-- Logs begin at Thu 2022-05-19 13:07:18 UTC, end at Thu 2022-05-19 14:15:18 UTC. -- May 19 13:39:14.965352 dv-cinder-784815-14-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 13:39:14.974024 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 13:39:14.976080 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 13:39:14 2022] *** May 19 13:39:14.976080 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 13:39:14.976080 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 13:39:14.976080 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: nodename: dv-cinder-784815-14-iscsi May 19 13:39:14.976080 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: machine: x86_64 May 19 13:39:14.976080 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: clock source: unix May 19 13:39:14.976080 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: pcre jit disabled May 19 13:39:14.976080 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: detected number of CPU cores: 4 May 19 13:39:14.976080 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: current working directory: / May 19 13:39:14.976080 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: detected binary path: /usr/bin/uwsgi-core May 19 13:39:14.976080 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: your processes number limit is 47680 May 19 13:39:14.976080 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: your memory page size is 4096 bytes May 19 13:39:14.976693 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: detected max file descriptor number: 2048 May 19 13:39:14.976693 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: lock engine: pthread robust mutexes May 19 13:39:14.976693 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: thunder lock: enabled May 19 13:39:14.976693 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 13:39:14.976693 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 13:39:15.021885 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: Python main interpreter initialized at 0x55c63f5199c0 May 19 13:39:15.021885 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: python threads support enabled May 19 13:39:15.021885 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: your server socket listen backlog is limited to 100 connections May 19 13:39:15.021885 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: your mercy for graceful operations on workers is 90 seconds May 19 13:39:15.022244 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: mapped 403077 bytes (393 KB) for 2 cores May 19 13:39:15.022319 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: *** Operational MODE: preforking *** May 19 13:39:15.022574 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: *** uWSGI is running in multiple interpreter mode *** May 19 13:39:15.022574 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: spawned uWSGI master process (pid: 86619) May 19 13:39:15.022779 dv-cinder-784815-14-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 13:39:15.023880 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: spawned uWSGI worker 1 (pid: 86630, cores: 1) May 19 13:39:15.023880 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: spawned uWSGI worker 2 (pid: 86631, cores: 1) May 19 13:39:15.023880 dv-cinder-784815-14-iscsi devstack@keystone.service[86619]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 13:39:20.715507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 13:39:20.715507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: from cryptography.utils import int_from_bytes May 19 13:39:20.716263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 13:39:20.716263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: from cryptography.utils import int_from_bytes May 19 13:39:20.730941 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 13:39:20.730941 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: from cryptography.utils import int_from_bytes May 19 13:39:20.731410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 13:39:20.731410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: from cryptography.utils import int_from_bytes May 19 13:39:20.766229 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 13:39:20.766426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 13:39:20.766622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] command line args: [] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 13:39:20.766914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 13:39:20.767101 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ================================================================================ {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 13:39:20.767358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] admin_token = **** {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.767605 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] config_dir = [] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.767794 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.767976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] config_source = [] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.768158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] debug = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.768359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.768870 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.768955 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.769157 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.769410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.769645 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] list_limit = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.769826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] log_config_append = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.770006 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.770200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] log_dir = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.770382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] log_file = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.770599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.770851 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.771731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.771989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.772171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.772348 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.772597 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.772845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.773082 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.773317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.773553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.773787 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.774037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.774285 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.774564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.775083 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.775280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] publish_errors = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.775504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.775736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.775923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.776171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.776407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.776643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] standard_threads = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.776878 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.777123 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.777300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.777477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] use_journal = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.777652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] use_json = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.777826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] use_stderr = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.778002 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] use_syslog = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.778234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.778494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.778736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.779025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.779270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.779522 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.779730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 13:39:20.779814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.779950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 13:39:20.780035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.780101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] command line args: [] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 13:39:20.780245 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.780376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 13:39:20.780452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.780585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ================================================================================ {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 13:39:20.780717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.780793 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] admin_token = **** {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.780960 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.781042 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] config_dir = [] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.781200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.781282 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.781408 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] config_source = [] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.781484 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.781556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] debug = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.781697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.781775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.781933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.782011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.782158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.782235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.782367 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.782507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.782647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.782731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.782889 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] list_limit = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.782981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.783043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] log_config_append = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.783189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.783278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.783405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] log_dir = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.783487 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.783548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] log_file = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.783691 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.783770 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.783904 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.783989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.784148 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.784230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.784401 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.784545 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.784683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.784809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.784936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.785062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.785198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.785332 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.785458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.785583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.785714 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.785838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.785963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.786096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.786227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.786352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.786477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.786599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.786808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.787090 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.787223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.787356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.787489 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.787622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.787622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.787870 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.787997 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.788123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] publish_errors = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.788247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.788379 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.788520 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.788653 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.788795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.788921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.789052 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.789186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.789312 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.789476 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.789612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.789738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] standard_threads = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.789863 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.789995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.790121 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.790253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.790253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.790495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.790631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] use_journal = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.790769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.790913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] use_json = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.791044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.791169 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] use_stderr = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.791169 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] use_syslog = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.791415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.791548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:39:20.791680 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.791812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.791945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.792070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.792194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.792338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.792464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.792589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.792762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.792889 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.793013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.793215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.793349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.793482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.793608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.793608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.793884 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.793884 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.794133 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.794133 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.794383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.794515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.794655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.794782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.794970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.794970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.795223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.795354 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.795481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.795616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.795742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.795742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.796999 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.797120 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.797212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.797212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.797350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.797350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.797539 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.797539 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.797717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.797800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.797871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.797985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.798058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.798058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.798246 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.798361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.798474 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.798547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.798661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.798734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.798855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.798966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.799120 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.799207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.799279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.799345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.799421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.799492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.799607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.799682 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.799796 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.799869 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.799951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.800023 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.800137 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.800137 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.800319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.800394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.800516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.800639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.800716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.800839 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.800915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.801030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.801103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.801174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.801296 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.801378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.801448 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.801524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.801600 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.801682 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.801759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.801830 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.801899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.801975 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.802052 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.802131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.802248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.802342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.802416 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.802581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.802701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.802789 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.802884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803004 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803162 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803456 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803456 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803580 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803870 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] auth.external = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.803940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.804015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.804086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.804235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.804310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.804380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.804450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.804525 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] auth.password = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.804596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.804677 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] auth.token = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.804758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.804829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.804899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.804969 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.805038 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.805116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.805195 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.805266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.805336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.805500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.805581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.805735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.805825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.805975 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.806055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.806217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] credential.caching = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.806296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.806466 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.806544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.806717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.806855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.806992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.807070 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.807231 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.807322 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.807470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.807549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.807709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.807786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.807960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.808046 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.808200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.808279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.808439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.808517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.808683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.808768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.808920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.808998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.809157 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.809239 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.809394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.809480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.809629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.809724 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.809864 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.809943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.810128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.810206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] federation.caching = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.810372 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.810459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.810615 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.810700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.810854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.810961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.811112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.811202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.811360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.811446 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.811600 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.811685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.811836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.811927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.812069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.812148 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.812299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.812379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.812529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.812617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.812763 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.812843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.812993 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.813071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.caching = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.813233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.813320 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.813468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.813548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.813707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.813786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.813942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.814029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.814190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.814297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.814424 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.814559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.814666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.814805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.814944 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.815063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] auth.external = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.815197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.815286 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.815437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.815583 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.815665 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.815818 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.815896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.816051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] auth.password = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.816129 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.816282 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] auth.token = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.816377 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.816523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.816605 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.816761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.816839 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.817016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.817097 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.817250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.817335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.817498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.817577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.817747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.817828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.818493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.818577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.818734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] credential.caching = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.818997 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.819187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.819264 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.819335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.819406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.819477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.819642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.819731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.819886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.819965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.820139 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.820217 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.820373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.820461 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.820623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.820702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.820872 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.820950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.821110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.821190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.821358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.821436 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.821608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.821687 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.821844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.821924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.822084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.822164 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.822317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.822395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.822557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.822637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] federation.caching = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.822797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.822891 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.823067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.823145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.823307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.823386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.823536 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.823613 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.823783 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.823863 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.824024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.824104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.824256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.824340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.824488 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.824567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.824739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.824820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.824975 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.825052 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.825213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.825291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.825446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.caching = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.825524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.825681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.825758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.825918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.825996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.826153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.826231 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.826389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.826467 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.826633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.826710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.826880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.826962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.827124 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.827204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.827376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.827453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.827614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.827696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.827845 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.827923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.828075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.828156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.828306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.828387 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.828542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.828619 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.828773 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.828852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.829020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.829102 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.829268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.829346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.829513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.829590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.829758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.829837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.830007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.830085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.830237 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.830322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.830467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.830546 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.830704 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.830782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.830957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.831040 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.831360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.831448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.831522 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.831593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.831664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.831735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.831883 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.831964 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.832112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.832190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.832359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.832438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.832596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.832676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.832829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.832906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.833065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.833151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.833303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.833382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.833549 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.833655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.834445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.834673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.834751 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.834823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.834925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.834999 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.835070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.835153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.835272 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.835358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] resource.caching = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.835509 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.835615 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.835745 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.835876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.835994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.836116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.836241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.836370 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.836473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.836625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.836705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.836864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.836942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.837114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.837199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.837354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.837435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.837595 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.837673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.837833 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] role.caching = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.837914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.838072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.838157 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] role.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.838303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.838381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.838555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.838639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.838786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.838892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.839058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.839137 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.839308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.839386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.839555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.839636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.839790 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.839869 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.840019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.840098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.840255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.840335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.840489 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.840569 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.840726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.840804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.840951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.841027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.841186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.841265 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.841432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.841511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.841677 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.841829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.841908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.842072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.842150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.842326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.842413 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.842580 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.842660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.842824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.842962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.843083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.843174 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.843318 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.843397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.843551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.843629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.843796 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.843877 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.844036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.844113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.844270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.844349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.844503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.844584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.844735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.844815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.844979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.845058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.845214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.845294 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.845448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.845526 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.845683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.845762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.845928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.846008 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.846158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.846236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.846387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.846464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] token.caching = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.846623 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.846703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.846879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] resource.caching = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.846962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.847125 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.847203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.847376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.847455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.847622 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.847701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.847862 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.847940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.848109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.848188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.848422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.848592 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.848669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.848826 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.848906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.849066 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.849176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.849358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.849437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] role.caching = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.849609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] role.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.849689 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.849843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.849921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.850095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.850173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.850343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.850422 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.850589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.850666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.850837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.850993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.connection = **** {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.851110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.851203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.851357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.851437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.851605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.851683 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.851853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.851938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.852084 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.852163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.852322 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.852399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.852561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.852647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.852801 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.852879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.853043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.853120 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.853280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.853436 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.853516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.853674 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.853751 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.853924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.854002 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.854173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.854253 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.854423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.854506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.854659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.854736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.854926 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.855010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.855155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.855233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.855328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86631) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 13:39:20.855407 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.855633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.855866 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.856098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.856332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.856583 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.856840 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.857087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.857329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.857565 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.857795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.858030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] token.caching = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.858267 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.858514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.858751 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.859038 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.859289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.859526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.859768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.860012 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.860259 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.860498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.860736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.860969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.861215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.861464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.861696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.861938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.862256 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.862520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.connection = **** {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.862767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.863038 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.863279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.863520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.863758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.863996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.864233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.864470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.864716 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.864976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.865221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.865458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.865706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.865943 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.866187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.866418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.866659 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.866912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:39:20.867085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 13:39:20.886615 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:20.886849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:20.887104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:20.892685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:20.892919 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:20.893144 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:20.895848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:20.903705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:20.925233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:20.925466 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:20.941985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:20.942326 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:20.980502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:20.993522 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.032281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.032504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.038647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.038899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.098571 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.098715 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.098801 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.098920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.099026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.099166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.099308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.099994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.100087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.100237 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.100329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.100456 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.124080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.124209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.124314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.124465 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.124561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.124640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.124715 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.124863 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.124958 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.125077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.125174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.125310 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.125396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.125482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.136680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.137162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.149927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.150159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.150384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.150606 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.150822 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.151069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.151329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.152175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.152411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.152640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.152866 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.153089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.153306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.153521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86631) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:39:21.168974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.169186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 13:39:21.169270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.169457 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 13:39:21.169553 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.169720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 13:39:21.169802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.169985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 13:39:21.170067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.170242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.170338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.170498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.170581 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.170752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.170833 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.171036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.182058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.182364 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.182917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.183211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.183712 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.183988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.184254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.184535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.191425 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.193130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.194733 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.195037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.195322 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.195718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.196003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.196267 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.196459 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.196541 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.196739 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.196822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.197021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.197104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.197318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.197419 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.197586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.197702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.197963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.198232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.198507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.198779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.199074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.199341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.225160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.226773 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.226896 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.227082 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.228418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.228697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.232388 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.232627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.232908 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.233951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.234227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.234506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.240306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.240586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.242054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.242336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.245200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.245492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.246041 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.246321 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.246596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.246881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.247169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.247248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.247437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.247520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.247738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.248018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.248290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.248561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.248827 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.249091 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.267191 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.267471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.272600 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.273572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.273964 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.276244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.276540 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.281512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.281628 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.285083 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.288000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.288477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.289432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.289734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.290019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.290579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.290874 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.296237 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.299885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.304509 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.305017 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.305360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.305645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.305921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.306201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.310193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.310598 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.310887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.311167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.319850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.321596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.321875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.324967 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.325249 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.325531 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.325622 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.325813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.325970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.326093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.326239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.335067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.336795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.337069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.340059 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.340160 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.340343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.340426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.340625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.340709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.340908 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.341000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.341198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.341281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.341473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.341762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.355141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.355422 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.355709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.355985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.356260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.356532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.356803 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.360956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.364246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.368354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.369972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.371924 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.372221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.372514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.376013 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.379290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.379937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.381675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.383064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.384677 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.384918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.385212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.385494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.385770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.386319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.386606 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.386895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.394507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.395064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.395427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.395756 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.395909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.399572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.399864 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.400145 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.400421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.403859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86631) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.404142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.404432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.404712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.404986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.405281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.405555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.405825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.406103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.406374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.406651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.406939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.407217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.407485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86631) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.409579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.409864 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.410141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.419193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:39:21.419473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.419749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.420027 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.420299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.420575 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.420853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.421125 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.421401 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.421670 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.421947 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.422214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.422476 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.422735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:39:21.448063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c63f5199c0 pid: 86631 (default app) May 19 13:39:21.451431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19bc70-c401-4335-a18e-cbab4efbc345 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:21.451730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19bc70-c401-4335-a18e-cbab4efbc345 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:21.452011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19bc70-c401-4335-a18e-cbab4efbc345 None None] PATH_INFO: `/v3` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:21.456236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Thu May 19 13:39:21 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:21.463024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c63f5199c0 pid: 86630 (default app) May 19 13:39:29.511662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd48633-96de-4941-b21b-5875a9c20f9f None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:29.512213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd48633-96de-4941-b21b-5875a9c20f9f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:29.512213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd48633-96de-4941-b21b-5875a9c20f9f None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:29.513557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 2/2] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:39:29.521426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031fd9-38ef-4f6c-a652-7e46168df09b None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:29.522182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031fd9-38ef-4f6c-a652-7e46168df09b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:29.522472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031fd9-38ef-4f6c-a652-7e46168df09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:29.589642 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: INFO dbcounter [None req-e7031fd9-38ef-4f6c-a652-7e46168df09b None None] Registered counter for database keystone May 19 13:39:29.611356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG oslo_db.sqlalchemy.engines [None req-e7031fd9-38ef-4f6c-a652-7e46168df09b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86630) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 13:39:29.618514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writer thread running {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 13:39:29.792614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG passlib.handlers.bcrypt [None req-e7031fd9-38ef-4f6c-a652-7e46168df09b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86630) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 13:39:29.792949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG passlib.handlers.bcrypt [None req-e7031fd9-38ef-4f6c-a652-7e46168df09b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86630) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 13:39:29.847539 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e7031fd9-38ef-4f6c-a652-7e46168df09b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:29.849692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e7031fd9-38ef-4f6c-a652-7e46168df09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:29.972103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1/3] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 13:39:30.154794 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd48633-96de-4941-b21b-5875a9c20f9f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:30.223252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: INFO dbcounter [None req-5fd48633-96de-4941-b21b-5875a9c20f9f None None] Registered counter for database keystone May 19 13:39:30.243043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG oslo_db.sqlalchemy.engines [None req-5fd48633-96de-4941-b21b-5875a9c20f9f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86631) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 13:39:30.249804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writer thread running {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 13:39:30.369346 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd48633-96de-4941-b21b-5875a9c20f9f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:30.373884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cecae-bf3b-4235-b5f1-5f615c04744c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:30.374546 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cecae-bf3b-4235-b5f1-5f615c04744c admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.374831 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cecae-bf3b-4235-b5f1-5f615c04744c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.375136 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cecae-bf3b-4235-b5f1-5f615c04744c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.383730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cecae-bf3b-4235-b5f1-5f615c04744c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:30.434828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e50cecae-bf3b-4235-b5f1-5f615c04744c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 13:39:30.435143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cecae-bf3b-4235-b5f1-5f615c04744c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:30.445906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 3/4] 10.222.0.52 () {58 vars in 1324 bytes} [Thu May 19 13:39:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:39:30.451884 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7031fd9-38ef-4f6c-a652-7e46168df09b None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:30.467053 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7031fd9-38ef-4f6c-a652-7e46168df09b None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:30.471430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64673b-cfe7-4866-ae41-e5bc42815e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:30.472527 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64673b-cfe7-4866-ae41-e5bc42815e9b admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.472814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64673b-cfe7-4866-ae41-e5bc42815e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.473090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64673b-cfe7-4866-ae41-e5bc42815e9b admin admin] PATH_INFO: `/v3/projects/2f0d02746fbd4bebabb402313125d4b8` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.474750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64673b-cfe7-4866-ae41-e5bc42815e9b admin admin] RBAC: Authorizing `identity:get_project(project_id=2f0d02746fbd4bebabb402313125d4b8)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:30.527535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eb64673b-cfe7-4866-ae41-e5bc42815e9b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 13:39:30.528076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64673b-cfe7-4866-ae41-e5bc42815e9b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:30.532395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 2/5] 10.222.0.52 () {56 vars in 1207 bytes} [Thu May 19 13:39:30 2022] GET /identity/v3/projects/2f0d02746fbd4bebabb402313125d4b8 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:32.732679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a069-e3b5-44fa-9177-454bf42e1b34 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:32.733189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a069-e3b5-44fa-9177-454bf42e1b34 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:32.733189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a069-e3b5-44fa-9177-454bf42e1b34 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:32.737285 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 4/6] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:32.743308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7687-7af2-4763-a243-d180198f8e81 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:32.743740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7687-7af2-4763-a243-d180198f8e81 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:32.743856 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7687-7af2-4763-a243-d180198f8e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:32.762695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-9bbb7687-7af2-4763-a243-d180198f8e81 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:32.764229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9bbb7687-7af2-4763-a243-d180198f8e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:32.772130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 3/7] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 13:39:32.794030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a069-e3b5-44fa-9177-454bf42e1b34 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:32.806557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a069-e3b5-44fa-9177-454bf42e1b34 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:32.810655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0756a-7e0b-4917-89cf-a7ddb8259947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:32.811310 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0756a-7e0b-4917-89cf-a7ddb8259947 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:32.811593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0756a-7e0b-4917-89cf-a7ddb8259947 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:32.811868 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0756a-7e0b-4917-89cf-a7ddb8259947 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:32.819520 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0756a-7e0b-4917-89cf-a7ddb8259947 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:32.821268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0756a-7e0b-4917-89cf-a7ddb8259947 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:32.829885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 5/8] 10.222.0.52 () {58 vars in 1324 bytes} [Thu May 19 13:39:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 13:39:32.835138 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb7687-7af2-4763-a243-d180198f8e81 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:32.844093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb7687-7af2-4763-a243-d180198f8e81 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:32.848500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5495a3-fafb-4204-97fa-fb2b29a1a65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:32.850398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5495a3-fafb-4204-97fa-fb2b29a1a65a admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:32.850684 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5495a3-fafb-4204-97fa-fb2b29a1a65a admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:32.850996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5495a3-fafb-4204-97fa-fb2b29a1a65a admin admin] PATH_INFO: `/v3/users/ea5484f975734ad68c3decb7b2d0c713` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:32.852423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5495a3-fafb-4204-97fa-fb2b29a1a65a admin admin] RBAC: Authorizing `identity:get_user(user_id=ea5484f975734ad68c3decb7b2d0c713)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:32.854239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5495a3-fafb-4204-97fa-fb2b29a1a65a admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:32.855844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 4/9] 10.222.0.52 () {56 vars in 1198 bytes} [Thu May 19 13:39:32 2022] GET /identity/v3/users/ea5484f975734ad68c3decb7b2d0c713 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:35.122084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5274ff-c578-4ff3-8c62-15944404ab2a None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.122615 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5274ff-c578-4ff3-8c62-15944404ab2a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.122805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5274ff-c578-4ff3-8c62-15944404ab2a None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.124126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 6/10] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:35.129349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277976e-e2fe-4c90-a1d9-5c5ad537354c None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.129789 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277976e-e2fe-4c90-a1d9-5c5ad537354c None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.129912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277976e-e2fe-4c90-a1d9-5c5ad537354c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.147238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-b277976e-e2fe-4c90-a1d9-5c5ad537354c None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:35.148393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-b277976e-e2fe-4c90-a1d9-5c5ad537354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:35.163283 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431948e-d483-4373-aba7-d2a55dbbbdde None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.163573 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431948e-d483-4373-aba7-d2a55dbbbdde None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.163850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431948e-d483-4373-aba7-d2a55dbbbdde None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.165186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 7/11] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:39:35.170029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ef4f4-231e-46d5-b820-531153f896d3 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.170385 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ef4f4-231e-46d5-b820-531153f896d3 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.170729 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ef4f4-231e-46d5-b820-531153f896d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.211162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG passlib.handlers.bcrypt [None req-de6ef4f4-231e-46d5-b820-531153f896d3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86631) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 13:39:35.211592 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG passlib.handlers.bcrypt [None req-de6ef4f4-231e-46d5-b820-531153f896d3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86631) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 13:39:35.228226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 5/12] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:35.251373 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277976e-e2fe-4c90-a1d9-5c5ad537354c None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:35.259554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-de6ef4f4-231e-46d5-b820-531153f896d3 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:35.261153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-de6ef4f4-231e-46d5-b820-531153f896d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:35.275434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:35.355842 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277976e-e2fe-4c90-a1d9-5c5ad537354c None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:35.361889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 8/13] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:39:35.383902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ef4f4-231e-46d5-b820-531153f896d3 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:35.392776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58752b10-045e-41f9-a1b4-0fa7bae805de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:35.393577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58752b10-045e-41f9-a1b4-0fa7bae805de None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.393860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58752b10-045e-41f9-a1b4-0fa7bae805de None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.394138 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58752b10-045e-41f9-a1b4-0fa7bae805de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.407412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58752b10-045e-41f9-a1b4-0fa7bae805de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-58752b10-045e-41f9-a1b4-0fa7bae805de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:39:35.419089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:35.420242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:35.421072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:35.421072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:35.421072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:35.421072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:35.421072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:39:35.421072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:39:35.421072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:39:35.421072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:39:35.421072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:39:35.421072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:39:35.421072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:39:35.421072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:39:35.421072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 6/14] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:39:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:39:35.426226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58752b10-045e-41f9-a1b4-0fa7bae805de None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:35.599701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ef4f4-231e-46d5-b820-531153f896d3 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:35.602037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:35.622370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:35.640924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58752b10-045e-41f9-a1b4-0fa7bae805de None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:35.641439 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee81fb-cf2a-48a4-ab83-99bb9008ed28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:35.642359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee81fb-cf2a-48a4-ab83-99bb9008ed28 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.642644 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee81fb-cf2a-48a4-ab83-99bb9008ed28 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.642943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee81fb-cf2a-48a4-ab83-99bb9008ed28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.676055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670d2d8-1692-47e5-88ed-7f54cd209e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:35.676715 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670d2d8-1692-47e5-88ed-7f54cd209e29 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.677029 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670d2d8-1692-47e5-88ed-7f54cd209e29 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.677367 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670d2d8-1692-47e5-88ed-7f54cd209e29 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.678122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670d2d8-1692-47e5-88ed-7f54cd209e29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:35.678426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670d2d8-1692-47e5-88ed-7f54cd209e29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:35.679829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670d2d8-1692-47e5-88ed-7f54cd209e29 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:35.693530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 7/15] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:39:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:39:35.699065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670d2d8-1692-47e5-88ed-7f54cd209e29 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:35.718072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee81fb-cf2a-48a4-ab83-99bb9008ed28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:35.719620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee81fb-cf2a-48a4-ab83-99bb9008ed28 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:35.797183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670d2d8-1692-47e5-88ed-7f54cd209e29 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:35.820221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 9/16] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:39:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:39:35.825963 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee81fb-cf2a-48a4-ab83-99bb9008ed28 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:35.828739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:35.832087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14296b8f-5f08-40ab-b998-73282b2f4a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:35.832963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14296b8f-5f08-40ab-b998-73282b2f4a94 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.833326 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14296b8f-5f08-40ab-b998-73282b2f4a94 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.834150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14296b8f-5f08-40ab-b998-73282b2f4a94 None admin] PATH_INFO: `/v3/users/ea5484f975734ad68c3decb7b2d0c713` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.835588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14296b8f-5f08-40ab-b998-73282b2f4a94 None admin] RBAC: Authorizing `identity:get_user(user_id=ea5484f975734ad68c3decb7b2d0c713)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:35.837056 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14296b8f-5f08-40ab-b998-73282b2f4a94 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:35.838666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 8/17] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:39:35 2022] GET /identity/v3/users/ea5484f975734ad68c3decb7b2d0c713 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:35.843863 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14296b8f-5f08-40ab-b998-73282b2f4a94 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:35.859558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:35.928161 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee81fb-cf2a-48a4-ab83-99bb9008ed28 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:35.946476 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14296b8f-5f08-40ab-b998-73282b2f4a94 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:35.961844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97ca15-5b56-4044-8e6e-7091a5eb6f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:35.962642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97ca15-5b56-4044-8e6e-7091a5eb6f4d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.962966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97ca15-5b56-4044-8e6e-7091a5eb6f4d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.963246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97ca15-5b56-4044-8e6e-7091a5eb6f4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.979990 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc9877-5b5a-4d06-a4dd-42df63b82ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:35.980782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc9877-5b5a-4d06-a4dd-42df63b82ecb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.981064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc9877-5b5a-4d06-a4dd-42df63b82ecb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.981341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc9877-5b5a-4d06-a4dd-42df63b82ecb None admin] PATH_INFO: `/v3/domains/default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.981722 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97ca15-5b56-4044-8e6e-7091a5eb6f4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:35.982515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc9877-5b5a-4d06-a4dd-42df63b82ecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:35.983207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97ca15-5b56-4044-8e6e-7091a5eb6f4d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:35.983970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc9877-5b5a-4d06-a4dd-42df63b82ecb None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:35.985549 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 9/18] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:39:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:35.991664 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc9877-5b5a-4d06-a4dd-42df63b82ecb None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-ff97ca15-5b56-4044-8e6e-7091a5eb6f4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:39:36.004412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:36.005906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:36.006709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:39:36.006709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:39:36.006709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:39:36.006709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:39:36.006709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 10/19] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:39:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:39:36.011923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97ca15-5b56-4044-8e6e-7091a5eb6f4d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:36.068978 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:36.074107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:36.094290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc9877-5b5a-4d06-a4dd-42df63b82ecb None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:36.113925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97ca15-5b56-4044-8e6e-7091a5eb6f4d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:36.128618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e985f-890e-4cc0-a6c2-556bee8b3f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:36.129274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e985f-890e-4cc0-a6c2-556bee8b3f36 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:36.129562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e985f-890e-4cc0-a6c2-556bee8b3f36 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:36.130174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e985f-890e-4cc0-a6c2-556bee8b3f36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:36.131304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e985f-890e-4cc0-a6c2-556bee8b3f36 None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.domain.id=default, user.id=ea5484f975734ad68c3decb7b2d0c713) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:36.131627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e985f-890e-4cc0-a6c2-556bee8b3f36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:36.133054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e985f-890e-4cc0-a6c2-556bee8b3f36 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:36.139700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 10/20] 10.222.0.52 () {56 vars in 1322 bytes} [Thu May 19 13:39:35 2022] GET /identity/v3/role_assignments?user.id=ea5484f975734ad68c3decb7b2d0c713&scope.domain.id=default&role.id=0893bb944dd94634accd3906aa391cd2 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:39:36.148493 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8fef4-8eba-413b-b625-fc3528b42b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:36.149181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8fef4-8eba-413b-b625-fc3528b42b8a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:36.149463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8fef4-8eba-413b-b625-fc3528b42b8a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:36.149739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8fef4-8eba-413b-b625-fc3528b42b8a None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:36.150477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8fef4-8eba-413b-b625-fc3528b42b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:36.150785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8fef4-8eba-413b-b625-fc3528b42b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:36.152238 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8fef4-8eba-413b-b625-fc3528b42b8a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:36.172325 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 11/21] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:39:39.075294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb763c-1b65-4b65-be80-ee70c92956b9 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:39.076349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb763c-1b65-4b65-be80-ee70c92956b9 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:39.076850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb763c-1b65-4b65-be80-ee70c92956b9 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:39.078447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 11/22] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:39.085330 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f7c26-67e1-49c4-81c0-8c1963501022 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:39.085849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f7c26-67e1-49c4-81c0-8c1963501022 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:39.085960 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f7c26-67e1-49c4-81c0-8c1963501022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:39.121361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-f60f7c26-67e1-49c4-81c0-8c1963501022 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:39.122579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f60f7c26-67e1-49c4-81c0-8c1963501022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:39.216529 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 12/23] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:39.256751 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb763c-1b65-4b65-be80-ee70c92956b9 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:39.302502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de108a-7857-4cbf-8101-865bad7b4f3a None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:39.303252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de108a-7857-4cbf-8101-865bad7b4f3a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:39.303734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de108a-7857-4cbf-8101-865bad7b4f3a None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:39.305364 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 13/24] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:39:39.320648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f91e89-3a83-46d0-8744-f3e08fc1ca45 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:39.321232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f91e89-3a83-46d0-8744-f3e08fc1ca45 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:39.321736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f91e89-3a83-46d0-8744-f3e08fc1ca45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:39.353856 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-05f91e89-3a83-46d0-8744-f3e08fc1ca45 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:39.355448 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-05f91e89-3a83-46d0-8744-f3e08fc1ca45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:39.428253 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:39.434898 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb763c-1b65-4b65-be80-ee70c92956b9 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:39.480248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff519c-5acf-4f88-af86-8861a9d93ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:39.481048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff519c-5acf-4f88-af86-8861a9d93ea2 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:39.481334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff519c-5acf-4f88-af86-8861a9d93ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:39.481610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff519c-5acf-4f88-af86-8861a9d93ea2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:39.483431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 14/25] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:39:39.485002 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576ff01-15ea-44c6-8330-af4a4089d9ca None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:39.485293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576ff01-15ea-44c6-8330-af4a4089d9ca None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:39.485572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576ff01-15ea-44c6-8330-af4a4089d9ca None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:39.486804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 15/26] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:39.495703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3251c04-ee6d-43b9-a264-ad5e4f47b817 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:39.496241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3251c04-ee6d-43b9-a264-ad5e4f47b817 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:39.496520 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3251c04-ee6d-43b9-a264-ad5e4f47b817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:39.503546 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff519c-5acf-4f88-af86-8861a9d93ea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:39.505042 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff519c-5acf-4f88-af86-8861a9d93ea2 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:39.527254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:39.528270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-e3251c04-ee6d-43b9-a264-ad5e4f47b817 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:39.529477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e3251c04-ee6d-43b9-a264-ad5e4f47b817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-deff519c-5acf-4f88-af86-8861a9d93ea2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:39:39.548874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:39.549727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:39.550508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:39:39.550508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:39:39.550508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:39:39.550508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:39:39.553827 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 12/27] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:39:39.554679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff519c-5acf-4f88-af86-8861a9d93ea2 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:39.634742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 16/28] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:39.635822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3251c04-ee6d-43b9-a264-ad5e4f47b817 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:39.712301 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff519c-5acf-4f88-af86-8861a9d93ea2 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:39.750056 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2093e-930a-4976-9048-8aab31098084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:39.751108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2093e-930a-4976-9048-8aab31098084 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:39.751530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2093e-930a-4976-9048-8aab31098084 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:39.751938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2093e-930a-4976-9048-8aab31098084 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:39.763875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2093e-930a-4976-9048-8aab31098084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-94f2093e-930a-4976-9048-8aab31098084 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:39:39.773316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:39.774235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:39.775098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:39.775098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:39.775098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:39.775098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:39.775098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:39:39.775098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:39:39.775098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:39:39.775098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:39:39.775098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:39:39.775098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:39:39.775098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:39:39.775098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:39:39.775668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3251c04-ee6d-43b9-a264-ad5e4f47b817 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:39.777629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 13/29] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:39:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:39:39.779186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0591-4587-4236-81ca-a807f6775f0d None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:39.779694 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0591-4587-4236-81ca-a807f6775f0d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:39.780200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0591-4587-4236-81ca-a807f6775f0d None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:39.781676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 14/30] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:39.782558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e0591-4587-4236-81ca-a807f6775f0d None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:39.789707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:39.828064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f4784-dd67-493d-92ec-7d77533fd262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:39.828742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f4784-dd67-493d-92ec-7d77533fd262 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:39.829027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f4784-dd67-493d-92ec-7d77533fd262 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:39.829305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f4784-dd67-493d-92ec-7d77533fd262 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:39.830036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f4784-dd67-493d-92ec-7d77533fd262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:39.830345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f4784-dd67-493d-92ec-7d77533fd262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:39.831779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f4784-dd67-493d-92ec-7d77533fd262 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:39.849396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 17/31] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:39.850797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f4784-dd67-493d-92ec-7d77533fd262 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:39.853327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:39.906061 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e0591-4587-4236-81ca-a807f6775f0d None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:39.943427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6242-c4c8-4858-9bf6-2541c3e9f749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:39.944588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6242-c4c8-4858-9bf6-2541c3e9f749 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:39.945083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6242-c4c8-4858-9bf6-2541c3e9f749 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:39.945629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6242-c4c8-4858-9bf6-2541c3e9f749 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:39.946570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f6242-c4c8-4858-9bf6-2541c3e9f749 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:39.948299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f6242-c4c8-4858-9bf6-2541c3e9f749 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:39.976336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f4784-dd67-493d-92ec-7d77533fd262 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.013167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d96d5f-c98b-4b9a-988e-d4810db78c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.014138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d96d5f-c98b-4b9a-988e-d4810db78c41 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.014566 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d96d5f-c98b-4b9a-988e-d4810db78c41 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.015021 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d96d5f-c98b-4b9a-988e-d4810db78c41 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.015991 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d96d5f-c98b-4b9a-988e-d4810db78c41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:40.016470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d96d5f-c98b-4b9a-988e-d4810db78c41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.018053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d96d5f-c98b-4b9a-988e-d4810db78c41 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:40.032959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 18/32] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:39:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:40.034738 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dfd8b-fe21-4ec8-a081-d0a5162af501 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.035273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dfd8b-fe21-4ec8-a081-d0a5162af501 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.035751 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dfd8b-fe21-4ec8-a081-d0a5162af501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.054292 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-342dfd8b-fe21-4ec8-a081-d0a5162af501 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:40.055865 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-342dfd8b-fe21-4ec8-a081-d0a5162af501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:40.080359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:40.086214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 15/33] 10.222.0.52 () {60 vars in 1131 bytes} [Thu May 19 13:39:39 2022] POST /identity/v3/roles => generated 219 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:40.088199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6242-c4c8-4858-9bf6-2541c3e9f749 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.175802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 19/34] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:40.176792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dfd8b-fe21-4ec8-a081-d0a5162af501 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.214592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6242-c4c8-4858-9bf6-2541c3e9f749 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.234822 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:40.261458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69002931-4035-453d-8967-3ff0d4c322e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.262473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69002931-4035-453d-8967-3ff0d4c322e7 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.262997 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69002931-4035-453d-8967-3ff0d4c322e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.263416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69002931-4035-453d-8967-3ff0d4c322e7 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.264298 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69002931-4035-453d-8967-3ff0d4c322e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.266236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69002931-4035-453d-8967-3ff0d4c322e7 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:40.301952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 16/35] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:39:40 2022] POST /identity/v3/projects => generated 301 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:40.304734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93334401-b9f4-4c5c-932a-7217288ae61c None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.305220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93334401-b9f4-4c5c-932a-7217288ae61c None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.305725 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93334401-b9f4-4c5c-932a-7217288ae61c None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.307594 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 17/36] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:40.308687 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93334401-b9f4-4c5c-932a-7217288ae61c None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.323731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dfd8b-fe21-4ec8-a081-d0a5162af501 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.366061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53497c-808d-486a-a9cb-c363a5adb764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.367116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53497c-808d-486a-a9cb-c363a5adb764 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.367468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53497c-808d-486a-a9cb-c363a5adb764 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.367817 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53497c-808d-486a-a9cb-c363a5adb764 None admin] PATH_INFO: `/v3/users/ea5484f975734ad68c3decb7b2d0c713` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.369796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53497c-808d-486a-a9cb-c363a5adb764 None admin] RBAC: Authorizing `identity:get_user(user_id=ea5484f975734ad68c3decb7b2d0c713)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.371668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53497c-808d-486a-a9cb-c363a5adb764 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:40.373926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 20/37] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:39:40 2022] GET /identity/v3/users/ea5484f975734ad68c3decb7b2d0c713 => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:40.375070 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15491f20-d098-4ea2-bbad-f9d2f8111242 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.375472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15491f20-d098-4ea2-bbad-f9d2f8111242 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.375792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15491f20-d098-4ea2-bbad-f9d2f8111242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.395565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-15491f20-d098-4ea2-bbad-f9d2f8111242 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:40.397481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-15491f20-d098-4ea2-bbad-f9d2f8111242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:40.426174 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:40.449212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93334401-b9f4-4c5c-932a-7217288ae61c None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.501673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 21/38] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:40.503177 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15491f20-d098-4ea2-bbad-f9d2f8111242 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.504425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ac381-5cb7-4b77-8b39-c89345b829fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.505823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ac381-5cb7-4b77-8b39-c89345b829fb None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.506368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ac381-5cb7-4b77-8b39-c89345b829fb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.506688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ac381-5cb7-4b77-8b39-c89345b829fb None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.507496 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ac381-5cb7-4b77-8b39-c89345b829fb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.508903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ac381-5cb7-4b77-8b39-c89345b829fb None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:40.532742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 18/39] 10.222.0.52 () {60 vars in 1131 bytes} [Thu May 19 13:39:40 2022] POST /identity/v3/roles => generated 225 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:40.535140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f57e14-1806-420b-bf9c-27d422f0fdff None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.535504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f57e14-1806-420b-bf9c-27d422f0fdff None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.535855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f57e14-1806-420b-bf9c-27d422f0fdff None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.538382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 19/40] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:40.539448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f57e14-1806-420b-bf9c-27d422f0fdff None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.589022 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:40.649545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15491f20-d098-4ea2-bbad-f9d2f8111242 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.688312 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ca1b1-d8b6-4c06-8f92-cde716df5a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.689039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ca1b1-d8b6-4c06-8f92-cde716df5a3f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.689327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ca1b1-d8b6-4c06-8f92-cde716df5a3f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.689606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ca1b1-d8b6-4c06-8f92-cde716df5a3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.690872 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ca1b1-d8b6-4c06-8f92-cde716df5a3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.691761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f57e14-1806-420b-bf9c-27d422f0fdff None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.692372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ca1b1-d8b6-4c06-8f92-cde716df5a3f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:40.694898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 22/41] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:39:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:40.700907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc6538-c82c-4876-8faf-6872ea4e9047 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.701203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc6538-c82c-4876-8faf-6872ea4e9047 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.701493 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc6538-c82c-4876-8faf-6872ea4e9047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.721990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-ebdc6538-c82c-4876-8faf-6872ea4e9047 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:40.725049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ebdc6538-c82c-4876-8faf-6872ea4e9047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:40.737600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8427b4-aec1-4969-95b4-9353ab1e722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.738475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8427b4-aec1-4969-95b4-9353ab1e722c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.738851 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8427b4-aec1-4969-95b4-9353ab1e722c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.739217 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8427b4-aec1-4969-95b4-9353ab1e722c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.740754 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8427b4-aec1-4969-95b4-9353ab1e722c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.742337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8427b4-aec1-4969-95b4-9353ab1e722c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:40.744146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 20/42] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:39:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:40.745434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8427b4-aec1-4969-95b4-9353ab1e722c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.745554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:40.802227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 23/43] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:40.803086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc6538-c82c-4876-8faf-6872ea4e9047 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.861683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:40.868168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8427b4-aec1-4969-95b4-9353ab1e722c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.903206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dd5ef-558b-4820-a970-d866012fe527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.904040 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc6538-c82c-4876-8faf-6872ea4e9047 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.904140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dd5ef-558b-4820-a970-d866012fe527 None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.904351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dd5ef-558b-4820-a970-d866012fe527 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.904640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dd5ef-558b-4820-a970-d866012fe527 None admin] PATH_INFO: `/v3/domains/default/users/ea5484f975734ad68c3decb7b2d0c713/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.906673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dd5ef-558b-4820-a970-d866012fe527 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ea5484f975734ad68c3decb7b2d0c713, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.908287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dd5ef-558b-4820-a970-d866012fe527 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:40.922167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 21/44] 10.222.0.52 () {58 vars in 1361 bytes} [Thu May 19 13:39:40 2022] PUT /identity/v3/domains/default/users/ea5484f975734ad68c3decb7b2d0c713/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:40.923590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dd5ef-558b-4820-a970-d866012fe527 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.944389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec06a7-59e4-40e9-8762-d93304d4c5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.945378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec06a7-59e4-40e9-8762-d93304d4c5a9 None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.945667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec06a7-59e4-40e9-8762-d93304d4c5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.945943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec06a7-59e4-40e9-8762-d93304d4c5a9 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.946686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec06a7-59e4-40e9-8762-d93304d4c5a9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.948805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec06a7-59e4-40e9-8762-d93304d4c5a9 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:41.041101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dd5ef-558b-4820-a970-d866012fe527 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:41.077721 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443a4e5-d710-4205-8aa1-04001f1e4151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:41.078413 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443a4e5-d710-4205-8aa1-04001f1e4151 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:41.078706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443a4e5-d710-4205-8aa1-04001f1e4151 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:41.079021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443a4e5-d710-4205-8aa1-04001f1e4151 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:41.079794 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443a4e5-d710-4205-8aa1-04001f1e4151 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:41.081188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443a4e5-d710-4205-8aa1-04001f1e4151 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:41.098916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 24/45] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:39:40 2022] POST /identity/v3/projects => generated 312 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:41.099182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 22/46] 10.222.0.52 () {60 vars in 1131 bytes} [Thu May 19 13:39:40 2022] POST /identity/v3/roles => generated 223 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:43.165094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0313c7-fd09-4e21-be2e-b3b18822d450 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:43.165620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0313c7-fd09-4e21-be2e-b3b18822d450 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:43.165694 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0313c7-fd09-4e21-be2e-b3b18822d450 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:43.167273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 25/47] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:43.172960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9b610-7484-43d4-9f8c-79d9de2c015d None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:43.173468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9b610-7484-43d4-9f8c-79d9de2c015d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:43.173578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9b610-7484-43d4-9f8c-79d9de2c015d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:43.192031 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-cfb9b610-7484-43d4-9f8c-79d9de2c015d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:43.193772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-cfb9b610-7484-43d4-9f8c-79d9de2c015d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:43.205791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:43.266267 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 23/48] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:43.290095 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0313c7-fd09-4e21-be2e-b3b18822d450 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:43.299731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:43.363944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa82ef7-f2b7-436b-9a7e-2748d68d02a7 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:43.364238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa82ef7-f2b7-436b-9a7e-2748d68d02a7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:43.365127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa82ef7-f2b7-436b-9a7e-2748d68d02a7 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:43.366499 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 24/49] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:39:43.371783 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cef78-55dd-4126-9bf7-7856f553fc07 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:43.372078 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cef78-55dd-4126-9bf7-7856f553fc07 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:43.372360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cef78-55dd-4126-9bf7-7856f553fc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:43.388759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-049cef78-55dd-4126-9bf7-7856f553fc07 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:43.389951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-049cef78-55dd-4126-9bf7-7856f553fc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:43.397180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0313c7-fd09-4e21-be2e-b3b18822d450 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:43.432017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2a838-1217-4e7c-921b-72d4af8b5765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:43.432735 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2a838-1217-4e7c-921b-72d4af8b5765 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:43.433015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2a838-1217-4e7c-921b-72d4af8b5765 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:43.433289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2a838-1217-4e7c-921b-72d4af8b5765 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:43.445177 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2a838-1217-4e7c-921b-72d4af8b5765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-66c2a838-1217-4e7c-921b-72d4af8b5765 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:39:43.454319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:43.455248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:43.456005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:43.456005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:43.456005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:43.456005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:43.456005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:39:43.456005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:39:43.456005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:39:43.456005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:39:43.456005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:39:43.456005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:39:43.456005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:39:43.456005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:39:43.456005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 26/50] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:39:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:39:43.461783 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 25/51] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:39:43.461933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2a838-1217-4e7c-921b-72d4af8b5765 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:43.486291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cef78-55dd-4126-9bf7-7856f553fc07 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:43.531098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:43.562449 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:43.566881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2a838-1217-4e7c-921b-72d4af8b5765 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:43.585657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cef78-55dd-4126-9bf7-7856f553fc07 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:43.601138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1b2cc-1a39-4ee8-8d3d-1bbdcc1502b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:43.601805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1b2cc-1a39-4ee8-8d3d-1bbdcc1502b5 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:43.602089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1b2cc-1a39-4ee8-8d3d-1bbdcc1502b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:43.602366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1b2cc-1a39-4ee8-8d3d-1bbdcc1502b5 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:43.603145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1b2cc-1a39-4ee8-8d3d-1bbdcc1502b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:43.603454 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1b2cc-1a39-4ee8-8d3d-1bbdcc1502b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:43.604965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1b2cc-1a39-4ee8-8d3d-1bbdcc1502b5 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:43.614582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 27/52] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:39:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:39:43.620024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1b2cc-1a39-4ee8-8d3d-1bbdcc1502b5 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:43.620142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9225b-3f86-4e10-a97a-1172dd637424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:43.621098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9225b-3f86-4e10-a97a-1172dd637424 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:43.621386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9225b-3f86-4e10-a97a-1172dd637424 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:43.621661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9225b-3f86-4e10-a97a-1172dd637424 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:43.622828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9225b-3f86-4e10-a97a-1172dd637424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:43.624287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9225b-3f86-4e10-a97a-1172dd637424 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:43.625880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 26/53] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:39:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:39:43.631512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9225b-3f86-4e10-a97a-1172dd637424 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:43.717750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1b2cc-1a39-4ee8-8d3d-1bbdcc1502b5 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:43.731534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9225b-3f86-4e10-a97a-1172dd637424 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:43.744248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:43.752721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e15f7-bb78-4e70-ae8b-aa8ed584c273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:43.753588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e15f7-bb78-4e70-ae8b-aa8ed584c273 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:43.753874 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e15f7-bb78-4e70-ae8b-aa8ed584c273 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:43.754153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e15f7-bb78-4e70-ae8b-aa8ed584c273 None admin] PATH_INFO: `/v3/users/ea5484f975734ad68c3decb7b2d0c713` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:43.755510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e15f7-bb78-4e70-ae8b-aa8ed584c273 None admin] RBAC: Authorizing `identity:get_user(user_id=ea5484f975734ad68c3decb7b2d0c713)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:43.756949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e15f7-bb78-4e70-ae8b-aa8ed584c273 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:43.758796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 28/54] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:39:43 2022] GET /identity/v3/users/ea5484f975734ad68c3decb7b2d0c713 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:39:43.763944 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e15f7-bb78-4e70-ae8b-aa8ed584c273 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:43.765399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed11b4-b37c-473a-93d6-67c7879e73e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:43.766063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed11b4-b37c-473a-93d6-67c7879e73e2 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:43.766351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed11b4-b37c-473a-93d6-67c7879e73e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:43.766629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed11b4-b37c-473a-93d6-67c7879e73e2 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:43.767374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed11b4-b37c-473a-93d6-67c7879e73e2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:43.768801 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed11b4-b37c-473a-93d6-67c7879e73e2 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:43.783195 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:43.796203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 27/55] 10.222.0.52 () {60 vars in 1140 bytes} [Thu May 19 13:39:43 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:39:43.869629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e15f7-bb78-4e70-ae8b-aa8ed584c273 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:43.905438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f6589-eca2-4660-a2af-9f130d8c3511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:43.906247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f6589-eca2-4660-a2af-9f130d8c3511 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:43.906534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f6589-eca2-4660-a2af-9f130d8c3511 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:43.906814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f6589-eca2-4660-a2af-9f130d8c3511 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:43.908160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f6589-eca2-4660-a2af-9f130d8c3511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:43.909617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f6589-eca2-4660-a2af-9f130d8c3511 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:43.911852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 29/56] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:39:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:39:43.917766 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed11b4-b37c-473a-93d6-67c7879e73e2 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:44.025544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed11b4-b37c-473a-93d6-67c7879e73e2 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:44.060085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9541c4-67ef-40a2-ad48-55a4564e46e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:44.060721 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9541c4-67ef-40a2-ad48-55a4564e46e9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:44.061005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9541c4-67ef-40a2-ad48-55a4564e46e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:44.061282 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9541c4-67ef-40a2-ad48-55a4564e46e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:44.062113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9541c4-67ef-40a2-ad48-55a4564e46e9 None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.domain.id=default, user.id=ea5484f975734ad68c3decb7b2d0c713) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:44.062427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9541c4-67ef-40a2-ad48-55a4564e46e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:44.063879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9541c4-67ef-40a2-ad48-55a4564e46e9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:44.069824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 28/57] 10.222.0.52 () {56 vars in 1322 bytes} [Thu May 19 13:39:43 2022] GET /identity/v3/role_assignments?user.id=ea5484f975734ad68c3decb7b2d0c713&scope.domain.id=default&role.id=0893bb944dd94634accd3906aa391cd2 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:46.028968 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985bc780-3e9d-4b6e-bf97-999e293eecd2 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:46.029511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985bc780-3e9d-4b6e-bf97-999e293eecd2 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:46.029511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985bc780-3e9d-4b6e-bf97-999e293eecd2 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:46.030916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 30/58] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:46.036699 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d048d9-32c8-4374-807c-6027ad7c7f93 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:46.037235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d048d9-32c8-4374-807c-6027ad7c7f93 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:46.037339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d048d9-32c8-4374-807c-6027ad7c7f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:46.053466 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-49d048d9-32c8-4374-807c-6027ad7c7f93 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:46.054645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-49d048d9-32c8-4374-807c-6027ad7c7f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:46.108311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:46.119379 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 29/59] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:46.143436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985bc780-3e9d-4b6e-bf97-999e293eecd2 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:46.204723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:46.243231 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985bc780-3e9d-4b6e-bf97-999e293eecd2 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:46.276615 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131341da-4e74-4246-9e07-a4b2c1ccd075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:46.277395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131341da-4e74-4246-9e07-a4b2c1ccd075 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:46.277682 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131341da-4e74-4246-9e07-a4b2c1ccd075 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:46.277961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131341da-4e74-4246-9e07-a4b2c1ccd075 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:46.279223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131341da-4e74-4246-9e07-a4b2c1ccd075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:46.280593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131341da-4e74-4246-9e07-a4b2c1ccd075 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:46.282194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 31/60] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:39:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:46.288284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d048d9-32c8-4374-807c-6027ad7c7f93 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:46.383262 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d048d9-32c8-4374-807c-6027ad7c7f93 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:46.416484 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf58b9e-2a51-4d09-83fe-1921fad6593b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:46.417148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf58b9e-2a51-4d09-83fe-1921fad6593b None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:46.417434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf58b9e-2a51-4d09-83fe-1921fad6593b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:46.417710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf58b9e-2a51-4d09-83fe-1921fad6593b None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:46.418414 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf58b9e-2a51-4d09-83fe-1921fad6593b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:46.419891 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf58b9e-2a51-4d09-83fe-1921fad6593b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:46.445015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 30/61] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:39:46 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:49.737344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce599b0c-04bd-4c0e-88d7-7ae6ae836095 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:49.738411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce599b0c-04bd-4c0e-88d7-7ae6ae836095 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:49.738972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce599b0c-04bd-4c0e-88d7-7ae6ae836095 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:49.741225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 32/62] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:49.747627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdeecc1-654a-48e3-ad1b-39086d67303f None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:49.748601 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdeecc1-654a-48e3-ad1b-39086d67303f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:49.749106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdeecc1-654a-48e3-ad1b-39086d67303f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:49.772929 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-7fdeecc1-654a-48e3-ad1b-39086d67303f None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:49.774454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-7fdeecc1-654a-48e3-ad1b-39086d67303f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:49.830560 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:49.866298 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 31/63] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:39:49.903136 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce599b0c-04bd-4c0e-88d7-7ae6ae836095 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:49.936438 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75949f05-a928-4fec-8c5a-01091f79d69a None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:49.936739 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75949f05-a928-4fec-8c5a-01091f79d69a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:49.937015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75949f05-a928-4fec-8c5a-01091f79d69a None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:49.938842 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 32/64] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:49.945111 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec0a7f-eb67-4952-8128-988a94788b20 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:49.945405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec0a7f-eb67-4952-8128-988a94788b20 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:49.945684 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec0a7f-eb67-4952-8128-988a94788b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:49.964460 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-5fec0a7f-eb67-4952-8128-988a94788b20 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:49.965681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5fec0a7f-eb67-4952-8128-988a94788b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:50.032250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce599b0c-04bd-4c0e-88d7-7ae6ae836095 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:50.070778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:50.082209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 33/65] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:39:50.084610 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec773c60-6b35-48f4-9837-2501c91c03b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:50.085422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec773c60-6b35-48f4-9837-2501c91c03b5 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.085709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec773c60-6b35-48f4-9837-2501c91c03b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.085990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec773c60-6b35-48f4-9837-2501c91c03b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.097114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec773c60-6b35-48f4-9837-2501c91c03b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-ec773c60-6b35-48f4-9837-2501c91c03b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:39:50.105034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:50.105875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:50.106629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:50.106629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:50.106629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:50.106629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:50.106629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:39:50.106629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:39:50.106629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:39:50.106629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:39:50.106629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:39:50.106629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:39:50.106629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:39:50.106629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:39:50.106629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 33/66] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:39:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:39:50.113038 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec0a7f-eb67-4952-8128-988a94788b20 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:50.117611 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec773c60-6b35-48f4-9837-2501c91c03b5 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:50.230276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec773c60-6b35-48f4-9837-2501c91c03b5 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:50.257631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec0a7f-eb67-4952-8128-988a94788b20 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:50.266425 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dc78c-9d8c-4daa-ac10-4789bd989fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:50.267286 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:50.267988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dc78c-9d8c-4daa-ac10-4789bd989fcd None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.268492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dc78c-9d8c-4daa-ac10-4789bd989fcd None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.268977 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dc78c-9d8c-4daa-ac10-4789bd989fcd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.282651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638dc78c-9d8c-4daa-ac10-4789bd989fcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-638dc78c-9d8c-4daa-ac10-4789bd989fcd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:39:50.291114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:50.292019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:50.292785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:50.292785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:50.292785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:50.292785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:50.292785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:39:50.292785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:39:50.292785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:39:50.292785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:39:50.292785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:39:50.292785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:39:50.292785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:39:50.292785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:39:50.300473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 34/67] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:39:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:39:50.300662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2240866-7f92-4e49-a134-0ff62847268f None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.300924 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2240866-7f92-4e49-a134-0ff62847268f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.301508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2240866-7f92-4e49-a134-0ff62847268f None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.306964 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 35/68] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:50.308964 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2240866-7f92-4e49-a134-0ff62847268f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:50.310893 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a118a85-3544-4c83-adfa-76d1ce0e34aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:50.311601 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a118a85-3544-4c83-adfa-76d1ce0e34aa None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.311886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a118a85-3544-4c83-adfa-76d1ce0e34aa None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.312164 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a118a85-3544-4c83-adfa-76d1ce0e34aa None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.312935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a118a85-3544-4c83-adfa-76d1ce0e34aa None admin] RBAC: Adding query filter params (name=member) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:50.313245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a118a85-3544-4c83-adfa-76d1ce0e34aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:50.314957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a118a85-3544-4c83-adfa-76d1ce0e34aa None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:50.325775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 34/69] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:39:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:50.328166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e9bfa-2889-466e-9cb3-d57e59b0a992 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.328636 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e9bfa-2889-466e-9cb3-d57e59b0a992 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.329233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e9bfa-2889-466e-9cb3-d57e59b0a992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.350368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-a81e9bfa-2889-466e-9cb3-d57e59b0a992 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:50.351988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a81e9bfa-2889-466e-9cb3-d57e59b0a992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:50.376860 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:50.440327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2240866-7f92-4e49-a134-0ff62847268f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:50.448086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 35/70] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:50.449738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e9bfa-2889-466e-9cb3-d57e59b0a992 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:50.482838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb8d8f-6a11-4821-baa7-d0cfeb205016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:50.483545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb8d8f-6a11-4821-baa7-d0cfeb205016 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.483837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb8d8f-6a11-4821-baa7-d0cfeb205016 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.484227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb8d8f-6a11-4821-baa7-d0cfeb205016 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.484989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb8d8f-6a11-4821-baa7-d0cfeb205016 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:50.485300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb8d8f-6a11-4821-baa7-d0cfeb205016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:50.486682 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb8d8f-6a11-4821-baa7-d0cfeb205016 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:50.496298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 36/71] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:39:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:50.497577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3f611-c6ff-49a6-8a51-2efb72f521e4 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.497869 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3f611-c6ff-49a6-8a51-2efb72f521e4 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.498147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3f611-c6ff-49a6-8a51-2efb72f521e4 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.499704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 37/72] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:50.500098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3f611-c6ff-49a6-8a51-2efb72f521e4 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:50.557210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:50.561221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e9bfa-2889-466e-9cb3-d57e59b0a992 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:50.595086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76ca56-90c0-4172-808a-4afa5766074d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:50.595959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76ca56-90c0-4172-808a-4afa5766074d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.596242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76ca56-90c0-4172-808a-4afa5766074d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.596518 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76ca56-90c0-4172-808a-4afa5766074d None admin] PATH_INFO: `/v3/users/3bd06221895d4193813d7412e1e6a315` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.599085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3f611-c6ff-49a6-8a51-2efb72f521e4 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:50.612226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:50.612616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76ca56-90c0-4172-808a-4afa5766074d None admin] RBAC: Authorizing `identity:get_user(user_id=3bd06221895d4193813d7412e1e6a315)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:50.615165 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76ca56-90c0-4172-808a-4afa5766074d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:50.616860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 36/73] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:39:50 2022] GET /identity/v3/users/3bd06221895d4193813d7412e1e6a315 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:50.620602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f2c9c-fcc4-4a6b-a8bd-b2ec82877718 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.621288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f2c9c-fcc4-4a6b-a8bd-b2ec82877718 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.621576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f2c9c-fcc4-4a6b-a8bd-b2ec82877718 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.622847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 37/74] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:50.627156 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f2c9c-fcc4-4a6b-a8bd-b2ec82877718 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:50.637172 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb1121-534a-4812-875f-9181a260646a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:50.637958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb1121-534a-4812-875f-9181a260646a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.638241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb1121-534a-4812-875f-9181a260646a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.638511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb1121-534a-4812-875f-9181a260646a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.648767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb1121-534a-4812-875f-9181a260646a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-cbdb1121-534a-4812-875f-9181a260646a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:39:50.656702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:50.657561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:50.658301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:50.658301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:50.658301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:50.658301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:50.658301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:39:50.658301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:39:50.658301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:39:50.658301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:39:50.658301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:39:50.658301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:39:50.658301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:39:50.658301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:39:50.658301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 38/75] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:39:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:39:50.660361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae125737-c83a-41f7-aa7e-6af3d1a838a2 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.660361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae125737-c83a-41f7-aa7e-6af3d1a838a2 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.660361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae125737-c83a-41f7-aa7e-6af3d1a838a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.678674 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-ae125737-c83a-41f7-aa7e-6af3d1a838a2 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:50.680219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ae125737-c83a-41f7-aa7e-6af3d1a838a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:50.728292 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f2c9c-fcc4-4a6b-a8bd-b2ec82877718 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:50.748690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 39/76] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:50.750025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae125737-c83a-41f7-aa7e-6af3d1a838a2 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:50.764576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80636ad-701c-4b59-b676-3d757faf1a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:50.765474 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80636ad-701c-4b59-b676-3d757faf1a1e None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.765761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80636ad-701c-4b59-b676-3d757faf1a1e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.766037 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80636ad-701c-4b59-b676-3d757faf1a1e None admin] PATH_INFO: `/v3/users/ea5484f975734ad68c3decb7b2d0c713` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.767248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80636ad-701c-4b59-b676-3d757faf1a1e None admin] RBAC: Authorizing `identity:get_user(user_id=ea5484f975734ad68c3decb7b2d0c713)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:50.768696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80636ad-701c-4b59-b676-3d757faf1a1e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:50.775571 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 38/77] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:39:50 2022] GET /identity/v3/users/ea5484f975734ad68c3decb7b2d0c713 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:50.775571 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861bf80-c6e0-4414-a7c3-fea2426d9aad None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.775571 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861bf80-c6e0-4414-a7c3-fea2426d9aad None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.775571 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861bf80-c6e0-4414-a7c3-fea2426d9aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.788699 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f861bf80-c6e0-4414-a7c3-fea2426d9aad None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:50.789894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f861bf80-c6e0-4414-a7c3-fea2426d9aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:50.806720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:50.854757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae125737-c83a-41f7-aa7e-6af3d1a838a2 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:50.856559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 39/78] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:50.857094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:50.859733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861bf80-c6e0-4414-a7c3-fea2426d9aad None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:50.890814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e1bf3-3024-422c-a55e-b1dd4663683b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:50.891499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e1bf3-3024-422c-a55e-b1dd4663683b None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:50.891784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e1bf3-3024-422c-a55e-b1dd4663683b None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:50.892062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e1bf3-3024-422c-a55e-b1dd4663683b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:50.958693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e1bf3-3024-422c-a55e-b1dd4663683b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:50.960187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e1bf3-3024-422c-a55e-b1dd4663683b None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:50.969609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861bf80-c6e0-4414-a7c3-fea2426d9aad None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:51.003324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0be5da-61fa-409d-8657-ced8636aed8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:51.003990 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0be5da-61fa-409d-8657-ced8636aed8c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:51.004279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0be5da-61fa-409d-8657-ced8636aed8c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:51.004559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0be5da-61fa-409d-8657-ced8636aed8c None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:51.006020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0be5da-61fa-409d-8657-ced8636aed8c None admin] RBAC: Adding query filter params (name=member) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:51.006332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0be5da-61fa-409d-8657-ced8636aed8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:51.007733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0be5da-61fa-409d-8657-ced8636aed8c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:51.015772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 40/79] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:39:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:51.016565 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0be5da-61fa-409d-8657-ced8636aed8c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:51.034073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:51.061514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 40/80] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:39:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:39:51.062355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e1bf3-3024-422c-a55e-b1dd4663683b None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:51.069508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:51.115237 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0be5da-61fa-409d-8657-ced8636aed8c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:51.148317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7f372-31ad-4e21-ba9d-09271bbac0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:51.149106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7f372-31ad-4e21-ba9d-09271bbac0a2 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:51.149394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7f372-31ad-4e21-ba9d-09271bbac0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:51.149676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7f372-31ad-4e21-ba9d-09271bbac0a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:51.150849 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7f372-31ad-4e21-ba9d-09271bbac0a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:51.152309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7f372-31ad-4e21-ba9d-09271bbac0a2 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:51.154075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 41/81] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:39:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:51.154677 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7f372-31ad-4e21-ba9d-09271bbac0a2 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:51.169311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e1bf3-3024-422c-a55e-b1dd4663683b None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:51.207712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dba21e-3f4f-4d65-a498-976c6b11d37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:51.208410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dba21e-3f4f-4d65-a498-976c6b11d37f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:51.208694 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dba21e-3f4f-4d65-a498-976c6b11d37f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:51.208973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dba21e-3f4f-4d65-a498-976c6b11d37f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:51.238669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:51.255560 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7f372-31ad-4e21-ba9d-09271bbac0a2 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:51.277310 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dba21e-3f4f-4d65-a498-976c6b11d37f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:51.278767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dba21e-3f4f-4d65-a498-976c6b11d37f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:51.281350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:51.288301 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22808391-28c0-4be0-83a5-16f2efff4282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:51.289115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22808391-28c0-4be0-83a5-16f2efff4282 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:51.289406 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22808391-28c0-4be0-83a5-16f2efff4282 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:51.289690 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22808391-28c0-4be0-83a5-16f2efff4282 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:51.299955 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22808391-28c0-4be0-83a5-16f2efff4282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-22808391-28c0-4be0-83a5-16f2efff4282 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:39:51.308662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:51.309655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:51.310641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:51.310641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:51.310641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:51.310641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:51.310641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:39:51.310641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:39:51.310641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:39:51.310641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:39:51.310641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:39:51.310641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:39:51.310641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:39:51.310641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:39:51.310641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 42/82] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:39:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:39:51.311369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22808391-28c0-4be0-83a5-16f2efff4282 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:51.386786 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 41/83] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:39:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:39:51.387676 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dba21e-3f4f-4d65-a498-976c6b11d37f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:51.418247 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22808391-28c0-4be0-83a5-16f2efff4282 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:51.451954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577ad53-09ad-44d3-9acf-9e9880a4382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:51.452815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577ad53-09ad-44d3-9acf-9e9880a4382c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:51.453102 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577ad53-09ad-44d3-9acf-9e9880a4382c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:51.453376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577ad53-09ad-44d3-9acf-9e9880a4382c None admin] PATH_INFO: `/v3/users/3bd06221895d4193813d7412e1e6a315` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:51.454573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577ad53-09ad-44d3-9acf-9e9880a4382c None admin] RBAC: Authorizing `identity:get_user(user_id=3bd06221895d4193813d7412e1e6a315)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:51.456055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577ad53-09ad-44d3-9acf-9e9880a4382c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:51.457671 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 43/84] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:39:51 2022] GET /identity/v3/users/3bd06221895d4193813d7412e1e6a315 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:51.458572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577ad53-09ad-44d3-9acf-9e9880a4382c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:51.472467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:51.488419 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dba21e-3f4f-4d65-a498-976c6b11d37f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:51.491497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:51.522954 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361219a-2f4e-4a86-b990-525813dca0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:51.523807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361219a-2f4e-4a86-b990-525813dca0bf None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:51.524089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361219a-2f4e-4a86-b990-525813dca0bf None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:51.524366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361219a-2f4e-4a86-b990-525813dca0bf None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:51.525542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361219a-2f4e-4a86-b990-525813dca0bf None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:51.527008 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361219a-2f4e-4a86-b990-525813dca0bf None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:51.528693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 42/85] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:39:51 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:51.529448 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361219a-2f4e-4a86-b990-525813dca0bf None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:51.560002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577ad53-09ad-44d3-9acf-9e9880a4382c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:51.592650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ab51-396c-469e-84fb-f6625bf1bf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:51.593322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ab51-396c-469e-84fb-f6625bf1bf26 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:51.593610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ab51-396c-469e-84fb-f6625bf1bf26 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:51.593895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ab51-396c-469e-84fb-f6625bf1bf26 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:51.594603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140ab51-396c-469e-84fb-f6625bf1bf26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:51.596076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140ab51-396c-469e-84fb-f6625bf1bf26 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:51.616704 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 44/86] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:39:51 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:51.617569 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ab51-396c-469e-84fb-f6625bf1bf26 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:51.629379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361219a-2f4e-4a86-b990-525813dca0bf None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:51.670438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373239b9-0e41-4d8c-a2d6-c7b5ff135d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:51.671202 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373239b9-0e41-4d8c-a2d6-c7b5ff135d1a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:51.671504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373239b9-0e41-4d8c-a2d6-c7b5ff135d1a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:51.671775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373239b9-0e41-4d8c-a2d6-c7b5ff135d1a None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:51.672558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373239b9-0e41-4d8c-a2d6-c7b5ff135d1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:51.672875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373239b9-0e41-4d8c-a2d6-c7b5ff135d1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:51.674342 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373239b9-0e41-4d8c-a2d6-c7b5ff135d1a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:51.684189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 43/87] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:39:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:51.685163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373239b9-0e41-4d8c-a2d6-c7b5ff135d1a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:51.701778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:51.728272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:51.743752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ab51-396c-469e-84fb-f6625bf1bf26 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:51.786249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887bd00-fa7f-40ab-9a85-60135ec28705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:51.786986 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887bd00-fa7f-40ab-9a85-60135ec28705 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:51.787408 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887bd00-fa7f-40ab-9a85-60135ec28705 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:51.787757 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887bd00-fa7f-40ab-9a85-60135ec28705 None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:51.789114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887bd00-fa7f-40ab-9a85-60135ec28705 None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:51.790669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887bd00-fa7f-40ab-9a85-60135ec28705 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:51.797547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 45/88] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:39:51 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:51.797547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887bd00-fa7f-40ab-9a85-60135ec28705 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:51.812092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373239b9-0e41-4d8c-a2d6-c7b5ff135d1a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:51.858268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768c943-eb8a-4440-96ca-e0ab8edafa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:51.858922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768c943-eb8a-4440-96ca-e0ab8edafa20 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:51.859868 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768c943-eb8a-4440-96ca-e0ab8edafa20 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:51.859868 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768c943-eb8a-4440-96ca-e0ab8edafa20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:51.934975 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887bd00-fa7f-40ab-9a85-60135ec28705 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:51.953266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768c943-eb8a-4440-96ca-e0ab8edafa20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:51.955683 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768c943-eb8a-4440-96ca-e0ab8edafa20 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:51.973397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fda70e-6ab3-45ac-b308-fb4f9690282d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:51.974097 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fda70e-6ab3-45ac-b308-fb4f9690282d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:51.974383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fda70e-6ab3-45ac-b308-fb4f9690282d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:51.974939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fda70e-6ab3-45ac-b308-fb4f9690282d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:51.975602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fda70e-6ab3-45ac-b308-fb4f9690282d None admin] RBAC: Adding query filter params (role.id=a77e3ea4b46b4a3aaca8e814f53809b5, scope.project.id=d9fbb046f448445799c63c2b7d78bd93, user.id=3bd06221895d4193813d7412e1e6a315) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:51.976716 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fda70e-6ab3-45ac-b308-fb4f9690282d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:51.977369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fda70e-6ab3-45ac-b308-fb4f9690282d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:51.981441 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:51.984113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 46/89] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:39:51 2022] GET /identity/v3/role_assignments?user.id=3bd06221895d4193813d7412e1e6a315&scope.project.id=d9fbb046f448445799c63c2b7d78bd93&role.id=a77e3ea4b46b4a3aaca8e814f53809b5 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:51.991384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fda70e-6ab3-45ac-b308-fb4f9690282d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:52.002147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:52.111530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fda70e-6ab3-45ac-b308-fb4f9690282d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:52.156918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85cd13-31a7-46e7-a1c7-8207ba3fda1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:52.157818 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85cd13-31a7-46e7-a1c7-8207ba3fda1b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:52.158099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85cd13-31a7-46e7-a1c7-8207ba3fda1b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:52.158385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85cd13-31a7-46e7-a1c7-8207ba3fda1b None admin] PATH_INFO: `/v3/users/3bd06221895d4193813d7412e1e6a315` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:52.160922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85cd13-31a7-46e7-a1c7-8207ba3fda1b None admin] RBAC: Authorizing `identity:get_user(user_id=3bd06221895d4193813d7412e1e6a315)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:52.162959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85cd13-31a7-46e7-a1c7-8207ba3fda1b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:52.168458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 47/90] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:39:51 2022] GET /identity/v3/users/3bd06221895d4193813d7412e1e6a315 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:52.170517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85cd13-31a7-46e7-a1c7-8207ba3fda1b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:52.197217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 44/91] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:39:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:39:52.198026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768c943-eb8a-4440-96ca-e0ab8edafa20 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:52.303105 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:52.353183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85cd13-31a7-46e7-a1c7-8207ba3fda1b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:52.364592 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768c943-eb8a-4440-96ca-e0ab8edafa20 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:52.427979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4db6ad-f994-4586-8174-f8c089e45ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:52.428648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4db6ad-f994-4586-8174-f8c089e45ab9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:52.428938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4db6ad-f994-4586-8174-f8c089e45ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:52.429220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4db6ad-f994-4586-8174-f8c089e45ab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:52.430062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4db6ad-f994-4586-8174-f8c089e45ab9 None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=d9fbb046f448445799c63c2b7d78bd93, user.id=ea5484f975734ad68c3decb7b2d0c713) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:52.430378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4db6ad-f994-4586-8174-f8c089e45ab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:52.432123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4db6ad-f994-4586-8174-f8c089e45ab9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:52.438754 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 48/92] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:39:52 2022] GET /identity/v3/role_assignments?user.id=ea5484f975734ad68c3decb7b2d0c713&scope.project.id=d9fbb046f448445799c63c2b7d78bd93&role.id=0893bb944dd94634accd3906aa391cd2 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:52.439599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4db6ad-f994-4586-8174-f8c089e45ab9 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:52.442978 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb2eee-d1e1-4499-b968-3ba7d3fce06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:52.443726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb2eee-d1e1-4499-b968-3ba7d3fce06b None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:52.444018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb2eee-d1e1-4499-b968-3ba7d3fce06b None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:52.444297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb2eee-d1e1-4499-b968-3ba7d3fce06b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:52.451794 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:52.568476 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4db6ad-f994-4586-8174-f8c089e45ab9 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:52.603173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb2eee-d1e1-4499-b968-3ba7d3fce06b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:52.604564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb2eee-d1e1-4499-b968-3ba7d3fce06b None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:52.606727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:52.613874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a30736-37dc-4a85-9c92-4a6762484f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:52.615197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a30736-37dc-4a85-9c92-4a6762484f98 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:52.615670 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a30736-37dc-4a85-9c92-4a6762484f98 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:52.616104 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a30736-37dc-4a85-9c92-4a6762484f98 None admin] PATH_INFO: `/v3/projects/0bd9c3e5e8d34da582e7695992f2b206` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:52.618302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a30736-37dc-4a85-9c92-4a6762484f98 None admin] RBAC: Authorizing `identity:get_project(project_id=0bd9c3e5e8d34da582e7695992f2b206)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:52.620001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a30736-37dc-4a85-9c92-4a6762484f98 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:52.622218 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 49/93] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:39:52 2022] GET /identity/v3/projects/0bd9c3e5e8d34da582e7695992f2b206 => generated 312 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:52.631859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a30736-37dc-4a85-9c92-4a6762484f98 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:52.735748 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 45/94] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:39:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:39:52.744543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb2eee-d1e1-4499-b968-3ba7d3fce06b None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:52.788973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:52.835453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a30736-37dc-4a85-9c92-4a6762484f98 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:52.875620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb2eee-d1e1-4499-b968-3ba7d3fce06b None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:52.888354 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75773ddc-69df-4611-8cc2-56c5ab078106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:52.889025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75773ddc-69df-4611-8cc2-56c5ab078106 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:52.889776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75773ddc-69df-4611-8cc2-56c5ab078106 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:52.890066 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75773ddc-69df-4611-8cc2-56c5ab078106 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:52.891067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75773ddc-69df-4611-8cc2-56c5ab078106 None admin] RBAC: Adding query filter params (role.id=a77e3ea4b46b4a3aaca8e814f53809b5, scope.project.id=0bd9c3e5e8d34da582e7695992f2b206, user.id=3bd06221895d4193813d7412e1e6a315) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:52.891392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75773ddc-69df-4611-8cc2-56c5ab078106 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:52.892863 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75773ddc-69df-4611-8cc2-56c5ab078106 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:52.900347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 50/95] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:39:52 2022] GET /identity/v3/role_assignments?user.id=3bd06221895d4193813d7412e1e6a315&scope.project.id=0bd9c3e5e8d34da582e7695992f2b206&role.id=a77e3ea4b46b4a3aaca8e814f53809b5 => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:52.949145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380853d-e6d4-4b05-ba04-4d5ead3d6369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:52.950042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380853d-e6d4-4b05-ba04-4d5ead3d6369 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:52.950326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380853d-e6d4-4b05-ba04-4d5ead3d6369 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:52.950602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380853d-e6d4-4b05-ba04-4d5ead3d6369 None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:52.955714 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380853d-e6d4-4b05-ba04-4d5ead3d6369 None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:52.957184 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380853d-e6d4-4b05-ba04-4d5ead3d6369 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:52.959914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 46/96] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:39:52 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:39:52.965444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75773ddc-69df-4611-8cc2-56c5ab078106 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:53.066982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:53.125537 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75773ddc-69df-4611-8cc2-56c5ab078106 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:53.164015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa0053-92ff-4a9a-84e6-611466fb5da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:53.164672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa0053-92ff-4a9a-84e6-611466fb5da3 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:53.164957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa0053-92ff-4a9a-84e6-611466fb5da3 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:53.165236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa0053-92ff-4a9a-84e6-611466fb5da3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:53.166072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa0053-92ff-4a9a-84e6-611466fb5da3 None admin] RBAC: Adding query filter params (role.id=5c2125e2125d40bca6590d50afd677d1, scope.project.id=d9fbb046f448445799c63c2b7d78bd93, user.id=3bd06221895d4193813d7412e1e6a315) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:53.166383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa0053-92ff-4a9a-84e6-611466fb5da3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:53.167927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa0053-92ff-4a9a-84e6-611466fb5da3 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:53.178338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 51/97] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:39:52 2022] GET /identity/v3/role_assignments?user.id=3bd06221895d4193813d7412e1e6a315&scope.project.id=d9fbb046f448445799c63c2b7d78bd93&role.id=5c2125e2125d40bca6590d50afd677d1 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:54.653374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900afee-22b0-44b6-8134-cdfd90a7d152 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:54.653848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900afee-22b0-44b6-8134-cdfd90a7d152 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:54.653848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900afee-22b0-44b6-8134-cdfd90a7d152 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:54.655374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 47/98] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:54.660962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c20bae-0007-4acf-8db2-7b7dc9cf2731 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:54.661374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c20bae-0007-4acf-8db2-7b7dc9cf2731 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:54.661546 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c20bae-0007-4acf-8db2-7b7dc9cf2731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:54.687892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-03c20bae-0007-4acf-8db2-7b7dc9cf2731 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:54.689448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-03c20bae-0007-4acf-8db2-7b7dc9cf2731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:54.780353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 52/99] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:54.784593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6969f52-685b-4bb9-b0e4-ea006206be56 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:54.784882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6969f52-685b-4bb9-b0e4-ea006206be56 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:54.785162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6969f52-685b-4bb9-b0e4-ea006206be56 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:54.787242 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 48/100] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:54.792450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53feed9-2150-4b68-912b-e6cb53787527 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:54.792748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53feed9-2150-4b68-912b-e6cb53787527 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:54.793032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53feed9-2150-4b68-912b-e6cb53787527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:54.812258 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-d53feed9-2150-4b68-912b-e6cb53787527 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:54.813431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d53feed9-2150-4b68-912b-e6cb53787527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:54.829890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6969f52-685b-4bb9-b0e4-ea006206be56 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:54.904971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:54.933378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 53/101] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:54.968914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53feed9-2150-4b68-912b-e6cb53787527 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:55.012629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6969f52-685b-4bb9-b0e4-ea006206be56 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:55.014423 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:55.071116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c419447-970f-4ea9-ba31-58c9299a7c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:55.072277 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c419447-970f-4ea9-ba31-58c9299a7c37 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:55.072765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c419447-970f-4ea9-ba31-58c9299a7c37 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:55.073374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c419447-970f-4ea9-ba31-58c9299a7c37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:55.074778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c419447-970f-4ea9-ba31-58c9299a7c37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:55.076439 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c419447-970f-4ea9-ba31-58c9299a7c37 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:55.078391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 49/102] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:39:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:39:55.091756 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c419447-970f-4ea9-ba31-58c9299a7c37 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:55.195779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53feed9-2150-4b68-912b-e6cb53787527 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:55.240346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07dd2c-5f59-4ae9-8b5b-afa48e54718a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:55.241720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07dd2c-5f59-4ae9-8b5b-afa48e54718a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:55.242287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07dd2c-5f59-4ae9-8b5b-afa48e54718a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:55.242808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07dd2c-5f59-4ae9-8b5b-afa48e54718a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:55.263949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07dd2c-5f59-4ae9-8b5b-afa48e54718a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-9b07dd2c-5f59-4ae9-8b5b-afa48e54718a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:39:55.272442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:55.273274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:55.274018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:55.274018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:55.274018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:55.274018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:55.274018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:39:55.274018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:39:55.274018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:39:55.274018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:39:55.274018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:39:55.274018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:39:55.274018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:39:55.274018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:39:55.280477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 54/103] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:39:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:39:55.286581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07dd2c-5f59-4ae9-8b5b-afa48e54718a None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:55.362897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:55.386930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c419447-970f-4ea9-ba31-58c9299a7c37 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:55.441215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07dd2c-5f59-4ae9-8b5b-afa48e54718a None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:55.453486 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee41343-ebb2-47a1-84f0-1d095f109287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:55.454634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee41343-ebb2-47a1-84f0-1d095f109287 None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:55.455156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee41343-ebb2-47a1-84f0-1d095f109287 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:55.455619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee41343-ebb2-47a1-84f0-1d095f109287 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:55.456670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee41343-ebb2-47a1-84f0-1d095f109287 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:55.466901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee41343-ebb2-47a1-84f0-1d095f109287 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:55.476943 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa02b6-6ed2-4752-b82a-5dce47b59278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:55.477946 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa02b6-6ed2-4752-b82a-5dce47b59278 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:55.478491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa02b6-6ed2-4752-b82a-5dce47b59278 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:55.479018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa02b6-6ed2-4752-b82a-5dce47b59278 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:55.480013 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa02b6-6ed2-4752-b82a-5dce47b59278 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:55.480531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa02b6-6ed2-4752-b82a-5dce47b59278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:55.482112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa02b6-6ed2-4752-b82a-5dce47b59278 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:55.491600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 55/104] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:39:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:39:55.497640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07232c1d-e4ca-4835-9a3d-5ccb82f6f1a0 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:55.498449 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07232c1d-e4ca-4835-9a3d-5ccb82f6f1a0 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:55.498980 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07232c1d-e4ca-4835-9a3d-5ccb82f6f1a0 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:55.500643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 56/105] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:55.501574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07232c1d-e4ca-4835-9a3d-5ccb82f6f1a0 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:55.522118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 50/106] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:39:55 2022] POST /identity/v3/projects => generated 302 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:55.527254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc78f1-9fc4-41e5-b76c-7e6d79372633 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:55.527595 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc78f1-9fc4-41e5-b76c-7e6d79372633 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:55.527887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc78f1-9fc4-41e5-b76c-7e6d79372633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:55.560544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-e2bc78f1-9fc4-41e5-b76c-7e6d79372633 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:55.561798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e2bc78f1-9fc4-41e5-b76c-7e6d79372633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:55.594579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:55.679592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07232c1d-e4ca-4835-9a3d-5ccb82f6f1a0 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:55.696588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:55.773416 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 51/107] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:55.783626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151e059-cd5e-4ea6-b5f0-e99a2c7b10fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:55.784537 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151e059-cd5e-4ea6-b5f0-e99a2c7b10fe None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:55.784826 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151e059-cd5e-4ea6-b5f0-e99a2c7b10fe None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:55.785107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151e059-cd5e-4ea6-b5f0-e99a2c7b10fe None admin] PATH_INFO: `/v3/users/ff4f688d55724f64b648dda8b1fa5f46` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:55.812570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151e059-cd5e-4ea6-b5f0-e99a2c7b10fe None admin] RBAC: Authorizing `identity:get_user(user_id=ff4f688d55724f64b648dda8b1fa5f46)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:55.814152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151e059-cd5e-4ea6-b5f0-e99a2c7b10fe None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:55.816088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 57/108] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:39:55 2022] GET /identity/v3/users/ff4f688d55724f64b648dda8b1fa5f46 => generated 289 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:55.816458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc78f1-9fc4-41e5-b76c-7e6d79372633 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:55.823561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151e059-cd5e-4ea6-b5f0-e99a2c7b10fe None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:56.001027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc78f1-9fc4-41e5-b76c-7e6d79372633 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:56.002736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151e059-cd5e-4ea6-b5f0-e99a2c7b10fe None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:56.045068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea71f6-8d6c-4f3e-8f26-74308edbd3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:56.045666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea71f6-8d6c-4f3e-8f26-74308edbd3ec None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:56.045959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea71f6-8d6c-4f3e-8f26-74308edbd3ec None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:56.046078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45dbc3-dd67-4a22-84f7-565b6ef70697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:56.046255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea71f6-8d6c-4f3e-8f26-74308edbd3ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:56.047006 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45dbc3-dd67-4a22-84f7-565b6ef70697 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:56.047305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45dbc3-dd67-4a22-84f7-565b6ef70697 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:56.047587 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45dbc3-dd67-4a22-84f7-565b6ef70697 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:56.063695 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45dbc3-dd67-4a22-84f7-565b6ef70697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-5a45dbc3-dd67-4a22-84f7-565b6ef70697 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:39:56.075657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:56.076553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:56.077288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:56.077288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:56.077288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:56.077288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:56.077288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:39:56.077288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:39:56.077288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:39:56.077288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:39:56.077288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:39:56.077288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:39:56.077288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:39:56.077288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:39:56.077852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 52/109] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:39:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:39:56.081548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85ef52-5462-4ba5-89e6-081f49d39f60 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:56.081852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85ef52-5462-4ba5-89e6-081f49d39f60 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:56.082138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85ef52-5462-4ba5-89e6-081f49d39f60 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:56.085153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 53/110] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:56.086416 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85ef52-5462-4ba5-89e6-081f49d39f60 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:56.091752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:56.110280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:56.135831 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea71f6-8d6c-4f3e-8f26-74308edbd3ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:56.137405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea71f6-8d6c-4f3e-8f26-74308edbd3ec None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:56.255683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 58/111] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:39:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:39:56.255683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16933b6-b3c4-4a17-b770-32001c94a52e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:56.255683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16933b6-b3c4-4a17-b770-32001c94a52e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:56.255683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16933b6-b3c4-4a17-b770-32001c94a52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:56.264496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85ef52-5462-4ba5-89e6-081f49d39f60 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:56.289014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f16933b6-b3c4-4a17-b770-32001c94a52e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:56.290341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f16933b6-b3c4-4a17-b770-32001c94a52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:56.319094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5fe8b-831d-4e69-b90e-c66ebac9175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:56.320419 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5fe8b-831d-4e69-b90e-c66ebac9175e None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:56.321053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5fe8b-831d-4e69-b90e-c66ebac9175e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:56.321530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5fe8b-831d-4e69-b90e-c66ebac9175e None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:56.323037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5fe8b-831d-4e69-b90e-c66ebac9175e None admin] RBAC: Adding query filter params (name=member) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:56.323524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5fe8b-831d-4e69-b90e-c66ebac9175e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:56.325860 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5fe8b-831d-4e69-b90e-c66ebac9175e None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:56.336946 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 54/112] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:39:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:56.338043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5fe8b-831d-4e69-b90e-c66ebac9175e None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:56.374688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:56.378520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 59/113] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:56.382906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67b2f3-9b5b-454f-bbe3-73f87a4ed5ba None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:56.383449 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67b2f3-9b5b-454f-bbe3-73f87a4ed5ba None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:56.383866 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67b2f3-9b5b-454f-bbe3-73f87a4ed5ba None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:56.385432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 60/114] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:56.386345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67b2f3-9b5b-454f-bbe3-73f87a4ed5ba None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:56.436405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:56.466575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5fe8b-831d-4e69-b90e-c66ebac9175e None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:56.504198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28baa81-a115-4541-9d3d-dd23faa79f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:56.506042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28baa81-a115-4541-9d3d-dd23faa79f1d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:56.506553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28baa81-a115-4541-9d3d-dd23faa79f1d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:56.507126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28baa81-a115-4541-9d3d-dd23faa79f1d None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:56.508687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28baa81-a115-4541-9d3d-dd23faa79f1d None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:56.510507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28baa81-a115-4541-9d3d-dd23faa79f1d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:56.512674 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 55/115] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:39:56 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:56.518753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dfc9e-543a-4dbe-b8ae-179ef608d099 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:56.519233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dfc9e-543a-4dbe-b8ae-179ef608d099 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:56.519686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dfc9e-543a-4dbe-b8ae-179ef608d099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:56.526472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67b2f3-9b5b-454f-bbe3-73f87a4ed5ba None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:56.539014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-771dfc9e-543a-4dbe-b8ae-179ef608d099 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:56.540483 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-771dfc9e-543a-4dbe-b8ae-179ef608d099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:56.572025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffabec-7f19-49e9-ae2b-ec69a4e52806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:56.573249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffabec-7f19-49e9-ae2b-ec69a4e52806 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:56.573741 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffabec-7f19-49e9-ae2b-ec69a4e52806 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:56.574228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffabec-7f19-49e9-ae2b-ec69a4e52806 None admin] PATH_INFO: `/v3/users/3bd06221895d4193813d7412e1e6a315` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:56.575914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffabec-7f19-49e9-ae2b-ec69a4e52806 None admin] RBAC: Authorizing `identity:get_user(user_id=3bd06221895d4193813d7412e1e6a315)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:56.577695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffabec-7f19-49e9-ae2b-ec69a4e52806 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:56.579759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 61/116] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:39:56 2022] GET /identity/v3/users/3bd06221895d4193813d7412e1e6a315 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:56.581011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffabec-7f19-49e9-ae2b-ec69a4e52806 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:56.641051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 56/117] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:56.643563 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dfc9e-543a-4dbe-b8ae-179ef608d099 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:56.672033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:56.718350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffabec-7f19-49e9-ae2b-ec69a4e52806 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:56.743256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:56.761927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065227b-ca80-46ac-997c-ab40e5c02e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:56.763155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065227b-ca80-46ac-997c-ab40e5c02e19 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:56.763665 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065227b-ca80-46ac-997c-ab40e5c02e19 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:56.764170 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065227b-ca80-46ac-997c-ab40e5c02e19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:56.776508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065227b-ca80-46ac-997c-ab40e5c02e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:39:56.780006 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dfc9e-543a-4dbe-b8ae-179ef608d099 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-1065227b-ca80-46ac-997c-ab40e5c02e19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:39:56.784700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:56.785555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:56.786841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:56.786841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:56.786841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:56.786841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:56.786841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:39:56.786841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:39:56.786841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:39:56.786841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:39:56.786841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:39:56.786841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:39:56.786841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:39:56.786841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:39:56.786841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 62/118] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:39:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:39:56.792745 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065227b-ca80-46ac-997c-ab40e5c02e19 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:56.819087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf5893-c42a-4e73-aaf7-8d85894cd36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:56.820007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf5893-c42a-4e73-aaf7-8d85894cd36a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:56.820463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf5893-c42a-4e73-aaf7-8d85894cd36a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:56.820948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf5893-c42a-4e73-aaf7-8d85894cd36a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:56.821852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf5893-c42a-4e73-aaf7-8d85894cd36a None admin] RBAC: Adding query filter params (role.id=33252f8dd8084b9ea3fcaf16f4820401, scope.project.id=d9fbb046f448445799c63c2b7d78bd93, user.id=ff4f688d55724f64b648dda8b1fa5f46) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:56.822291 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf5893-c42a-4e73-aaf7-8d85894cd36a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:56.823756 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf5893-c42a-4e73-aaf7-8d85894cd36a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:56.838884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 57/119] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:39:56 2022] GET /identity/v3/role_assignments?user.id=ff4f688d55724f64b648dda8b1fa5f46&scope.project.id=d9fbb046f448445799c63c2b7d78bd93&role.id=33252f8dd8084b9ea3fcaf16f4820401 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:56.838884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf5893-c42a-4e73-aaf7-8d85894cd36a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:56.916515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065227b-ca80-46ac-997c-ab40e5c02e19 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:56.947676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:56.957481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e8389-971f-4da1-9f70-6f018fded95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:56.959264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e8389-971f-4da1-9f70-6f018fded95d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:56.959701 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e8389-971f-4da1-9f70-6f018fded95d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:56.960057 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e8389-971f-4da1-9f70-6f018fded95d None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:56.961424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e8389-971f-4da1-9f70-6f018fded95d None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:56.963028 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e8389-971f-4da1-9f70-6f018fded95d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:56.965187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 63/120] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:39:56 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:56.966630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d9cf2-f12b-4a9c-9a8d-d99945fc09b0 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:56.967009 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d9cf2-f12b-4a9c-9a8d-d99945fc09b0 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:56.967381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d9cf2-f12b-4a9c-9a8d-d99945fc09b0 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:56.969060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 64/121] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:56.974440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d9cf2-f12b-4a9c-9a8d-d99945fc09b0 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:56.988798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf5893-c42a-4e73-aaf7-8d85894cd36a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:57.037741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2575bf2-60ca-42cd-a514-545d9fe6112b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:57.038661 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2575bf2-60ca-42cd-a514-545d9fe6112b None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:57.038989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2575bf2-60ca-42cd-a514-545d9fe6112b None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:57.039301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2575bf2-60ca-42cd-a514-545d9fe6112b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:57.042337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:57.054721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2575bf2-60ca-42cd-a514-545d9fe6112b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-c2575bf2-60ca-42cd-a514-545d9fe6112b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:39:57.062497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:57.063392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:57.065094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:57.065094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:57.065094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:57.065094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:57.065094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:39:57.065094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:39:57.065094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:39:57.065094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:39:57.065094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:39:57.065094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:39:57.065094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:39:57.065094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:39:57.069246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 58/122] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:39:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:39:57.070192 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2575bf2-60ca-42cd-a514-545d9fe6112b None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:57.100093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d9cf2-f12b-4a9c-9a8d-d99945fc09b0 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:57.140840 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c43824-910a-4efe-abe8-ea5b4c7750dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:57.141822 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c43824-910a-4efe-abe8-ea5b4c7750dc None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:57.142218 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c43824-910a-4efe-abe8-ea5b4c7750dc None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:57.142604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c43824-910a-4efe-abe8-ea5b4c7750dc None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:57.143617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c43824-910a-4efe-abe8-ea5b4c7750dc None admin] RBAC: Adding query filter params (name=member) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:57.144144 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c43824-910a-4efe-abe8-ea5b4c7750dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:57.146547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c43824-910a-4efe-abe8-ea5b4c7750dc None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:57.156825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 65/123] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:39:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:57.163961 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5327263-44dd-46e9-830d-45345c0eca2f None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:57.164395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5327263-44dd-46e9-830d-45345c0eca2f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:57.164786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5327263-44dd-46e9-830d-45345c0eca2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:57.183831 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e5327263-44dd-46e9-830d-45345c0eca2f None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:57.185363 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e5327263-44dd-46e9-830d-45345c0eca2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:57.191902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2575bf2-60ca-42cd-a514-545d9fe6112b None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:57.238796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619efca-7b9b-4cc8-bc54-482137617ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:57.240046 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619efca-7b9b-4cc8-bc54-482137617ba6 None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:57.240765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619efca-7b9b-4cc8-bc54-482137617ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:57.241055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619efca-7b9b-4cc8-bc54-482137617ba6 None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93/users/3bd06221895d4193813d7412e1e6a315/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:57.244183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619efca-7b9b-4cc8-bc54-482137617ba6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9fbb046f448445799c63c2b7d78bd93, user_id=3bd06221895d4193813d7412e1e6a315, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:57.246389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619efca-7b9b-4cc8-bc54-482137617ba6 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:57.246988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:57.268372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 59/124] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:39:57 2022] PUT /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93/users/3bd06221895d4193813d7412e1e6a315/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:57.269555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619efca-7b9b-4cc8-bc54-482137617ba6 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:57.272272 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 66/125] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:57.279191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5327263-44dd-46e9-830d-45345c0eca2f None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:57.411463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:57.478694 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619efca-7b9b-4cc8-bc54-482137617ba6 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:57.488042 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5327263-44dd-46e9-830d-45345c0eca2f None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:57.525594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32825720-f0f0-4d9c-b32d-028cfd7d1170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:57.526271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32825720-f0f0-4d9c-b32d-028cfd7d1170 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:57.526560 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32825720-f0f0-4d9c-b32d-028cfd7d1170 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:57.526849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32825720-f0f0-4d9c-b32d-028cfd7d1170 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:57.527757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32825720-f0f0-4d9c-b32d-028cfd7d1170 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:57.528075 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32825720-f0f0-4d9c-b32d-028cfd7d1170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:57.529541 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32825720-f0f0-4d9c-b32d-028cfd7d1170 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:57.540416 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 60/126] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:39:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:57.540416 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32825720-f0f0-4d9c-b32d-028cfd7d1170 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:57.545511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889b7f2-e5ed-4e6f-81e1-ded120a30980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:57.546747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889b7f2-e5ed-4e6f-81e1-ded120a30980 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:57.547309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889b7f2-e5ed-4e6f-81e1-ded120a30980 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:57.547890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889b7f2-e5ed-4e6f-81e1-ded120a30980 None admin] PATH_INFO: `/v3/users/3bd06221895d4193813d7412e1e6a315` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:57.549426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5889b7f2-e5ed-4e6f-81e1-ded120a30980 None admin] RBAC: Authorizing `identity:get_user(user_id=3bd06221895d4193813d7412e1e6a315)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:57.551197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5889b7f2-e5ed-4e6f-81e1-ded120a30980 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:57.553772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 67/127] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:39:57 2022] GET /identity/v3/users/3bd06221895d4193813d7412e1e6a315 => generated 275 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:57.554918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889b7f2-e5ed-4e6f-81e1-ded120a30980 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:57.635386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:57.679350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32825720-f0f0-4d9c-b32d-028cfd7d1170 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:57.698328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:57.700804 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889b7f2-e5ed-4e6f-81e1-ded120a30980 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:57.722380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be69381-8d7e-4125-9d24-94ae091d471d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:57.723744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be69381-8d7e-4125-9d24-94ae091d471d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:57.724426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be69381-8d7e-4125-9d24-94ae091d471d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:57.724911 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be69381-8d7e-4125-9d24-94ae091d471d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:57.743983 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be69381-8d7e-4125-9d24-94ae091d471d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-5be69381-8d7e-4125-9d24-94ae091d471d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:39:57.752987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:39:57.753892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:39:57.755272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:39:57.755272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:39:57.755272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:39:57.755272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:39:57.755272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:39:57.755272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:39:57.755272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:39:57.755272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:39:57.755272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:39:57.755272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:39:57.755272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:39:57.755272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:39:57.758068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 61/128] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:39:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:39:57.763321 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be69381-8d7e-4125-9d24-94ae091d471d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:57.772290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2b783-ee35-438c-8dbb-d10b7f5edec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:57.773190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2b783-ee35-438c-8dbb-d10b7f5edec7 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:57.773479 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2b783-ee35-438c-8dbb-d10b7f5edec7 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:57.773758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2b783-ee35-438c-8dbb-d10b7f5edec7 None admin] PATH_INFO: `/v3/users/ea5484f975734ad68c3decb7b2d0c713` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:57.779695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2b783-ee35-438c-8dbb-d10b7f5edec7 None admin] RBAC: Authorizing `identity:get_user(user_id=ea5484f975734ad68c3decb7b2d0c713)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:57.781183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2b783-ee35-438c-8dbb-d10b7f5edec7 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:57.782903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 68/129] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:39:57 2022] GET /identity/v3/users/ea5484f975734ad68c3decb7b2d0c713 => generated 247 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:57.783670 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2b783-ee35-438c-8dbb-d10b7f5edec7 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:57.873132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be69381-8d7e-4125-9d24-94ae091d471d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:57.910673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0045396-d1bd-4997-83ae-2d5b05e05ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:57.911870 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0045396-d1bd-4997-83ae-2d5b05e05ce7 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:57.912258 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0045396-d1bd-4997-83ae-2d5b05e05ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:57.912640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0045396-d1bd-4997-83ae-2d5b05e05ce7 None admin] PATH_INFO: `/v3/projects/0bd9c3e5e8d34da582e7695992f2b206` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:57.913955 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0045396-d1bd-4997-83ae-2d5b05e05ce7 None admin] RBAC: Authorizing `identity:get_project(project_id=0bd9c3e5e8d34da582e7695992f2b206)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:57.915563 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0045396-d1bd-4997-83ae-2d5b05e05ce7 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:57.917390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 62/130] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:39:57 2022] GET /identity/v3/projects/0bd9c3e5e8d34da582e7695992f2b206 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:57.918252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0045396-d1bd-4997-83ae-2d5b05e05ce7 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:57.983354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:58.046653 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2b783-ee35-438c-8dbb-d10b7f5edec7 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:58.054441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0045396-d1bd-4997-83ae-2d5b05e05ce7 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:58.092292 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:58.119341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6393e6-0113-4f01-9530-dabdb27fc960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:58.120024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6393e6-0113-4f01-9530-dabdb27fc960 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:58.120310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6393e6-0113-4f01-9530-dabdb27fc960 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:58.120590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6393e6-0113-4f01-9530-dabdb27fc960 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:58.121615 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6393e6-0113-4f01-9530-dabdb27fc960 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:58.121922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6393e6-0113-4f01-9530-dabdb27fc960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:58.123559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6393e6-0113-4f01-9530-dabdb27fc960 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:58.132464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89811157-d513-47ea-a638-a5df237e443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:58.133341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89811157-d513-47ea-a638-a5df237e443d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:58.133628 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89811157-d513-47ea-a638-a5df237e443d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:58.133908 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89811157-d513-47ea-a638-a5df237e443d None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:58.135973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89811157-d513-47ea-a638-a5df237e443d None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:58.137421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89811157-d513-47ea-a638-a5df237e443d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:58.139157 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 63/131] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:39:57 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:58.140134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89811157-d513-47ea-a638-a5df237e443d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:58.146925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 69/132] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:39:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:58.147785 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6393e6-0113-4f01-9530-dabdb27fc960 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:58.300667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89811157-d513-47ea-a638-a5df237e443d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:58.317392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6393e6-0113-4f01-9530-dabdb27fc960 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:58.319959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:58.347070 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ca1e0-4025-4dfc-a45e-d3133d8f6491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:58.349678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ca1e0-4025-4dfc-a45e-d3133d8f6491 None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:58.350448 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ca1e0-4025-4dfc-a45e-d3133d8f6491 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:58.350972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ca1e0-4025-4dfc-a45e-d3133d8f6491 None admin] PATH_INFO: `/v3/projects/0bd9c3e5e8d34da582e7695992f2b206/users/3bd06221895d4193813d7412e1e6a315/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:58.354138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ca1e0-4025-4dfc-a45e-d3133d8f6491 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bd9c3e5e8d34da582e7695992f2b206, user_id=3bd06221895d4193813d7412e1e6a315, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:58.355810 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ca1e0-4025-4dfc-a45e-d3133d8f6491 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:58.357001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673c919-8ae6-43ed-a1d7-f9baf68a568d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:58.357900 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673c919-8ae6-43ed-a1d7-f9baf68a568d None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:58.358190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673c919-8ae6-43ed-a1d7-f9baf68a568d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:58.358471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673c919-8ae6-43ed-a1d7-f9baf68a568d None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93/users/ea5484f975734ad68c3decb7b2d0c713/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:58.360752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673c919-8ae6-43ed-a1d7-f9baf68a568d None admin] RBAC: Authorizing `identity:create_grant(project_id=d9fbb046f448445799c63c2b7d78bd93, user_id=ea5484f975734ad68c3decb7b2d0c713, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:58.363900 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673c919-8ae6-43ed-a1d7-f9baf68a568d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:58.374031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 64/133] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:39:58 2022] PUT /identity/v3/projects/0bd9c3e5e8d34da582e7695992f2b206/users/3bd06221895d4193813d7412e1e6a315/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:58.375323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ca1e0-4025-4dfc-a45e-d3133d8f6491 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:58.397383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 70/134] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:39:58 2022] PUT /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93/users/ea5484f975734ad68c3decb7b2d0c713/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:58.398475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71224f7-561a-48ff-a7bb-7d45f2b8a196 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:58.398800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71224f7-561a-48ff-a7bb-7d45f2b8a196 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:58.399106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71224f7-561a-48ff-a7bb-7d45f2b8a196 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:58.400364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 71/135] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:58.413394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c4f9-3a17-404c-9115-89dc3d279077 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:58.413730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c4f9-3a17-404c-9115-89dc3d279077 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:58.413973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c4f9-3a17-404c-9115-89dc3d279077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:58.468126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ca68c4f9-3a17-404c-9115-89dc3d279077 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:58.469395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ca68c4f9-3a17-404c-9115-89dc3d279077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:58.512862 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ca1e0-4025-4dfc-a45e-d3133d8f6491 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:58.558453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d6bbe-19e6-43b3-8c18-70e956ca02ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:58.559841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d6bbe-19e6-43b3-8c18-70e956ca02ee None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:58.560346 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d6bbe-19e6-43b3-8c18-70e956ca02ee None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:58.560816 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d6bbe-19e6-43b3-8c18-70e956ca02ee None admin] PATH_INFO: `/v3/users/3bd06221895d4193813d7412e1e6a315` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:58.567922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d6bbe-19e6-43b3-8c18-70e956ca02ee None admin] RBAC: Authorizing `identity:get_user(user_id=3bd06221895d4193813d7412e1e6a315)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:58.569783 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d6bbe-19e6-43b3-8c18-70e956ca02ee None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:58.571807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 65/136] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:39:58 2022] GET /identity/v3/users/3bd06221895d4193813d7412e1e6a315 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:58.579391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:58.581232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d6bbe-19e6-43b3-8c18-70e956ca02ee None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:58.632900 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 72/137] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:39:58.664401 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68c4f9-3a17-404c-9115-89dc3d279077 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:58.680492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:58.785337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d6bbe-19e6-43b3-8c18-70e956ca02ee None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:58.839056 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68c4f9-3a17-404c-9115-89dc3d279077 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:58.860802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af16457-4375-450d-939c-be63e43ca4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:58.861688 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af16457-4375-450d-939c-be63e43ca4c5 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:58.861979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af16457-4375-450d-939c-be63e43ca4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:58.862261 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af16457-4375-450d-939c-be63e43ca4c5 None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:58.863498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af16457-4375-450d-939c-be63e43ca4c5 None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:58.865440 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af16457-4375-450d-939c-be63e43ca4c5 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:58.873703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 66/138] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:39:58 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:58.874569 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af16457-4375-450d-939c-be63e43ca4c5 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:58.917201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d955b-9a2c-41c4-b29e-8e5fb331ea42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:58.918020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d955b-9a2c-41c4-b29e-8e5fb331ea42 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:58.918308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d955b-9a2c-41c4-b29e-8e5fb331ea42 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:58.918585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d955b-9a2c-41c4-b29e-8e5fb331ea42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:58.923628 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d955b-9a2c-41c4-b29e-8e5fb331ea42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:58.925103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d955b-9a2c-41c4-b29e-8e5fb331ea42 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:58.927418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 73/139] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:39:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:39:58.933195 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d955b-9a2c-41c4-b29e-8e5fb331ea42 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:58.985509 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:59.046683 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af16457-4375-450d-939c-be63e43ca4c5 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:59.058238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d955b-9a2c-41c4-b29e-8e5fb331ea42 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:59.066516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:59.117375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2e11e-3bbb-4fbd-9f7b-292887b3e8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:59.118282 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2e11e-3bbb-4fbd-9f7b-292887b3e8ff None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:59.118573 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2e11e-3bbb-4fbd-9f7b-292887b3e8ff None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:59.118881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2e11e-3bbb-4fbd-9f7b-292887b3e8ff None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93/users/3bd06221895d4193813d7412e1e6a315/roles/5c2125e2125d40bca6590d50afd677d1` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:59.121425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee960d44-5a03-441f-aa4e-012a294faf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:59.122096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee960d44-5a03-441f-aa4e-012a294faf93 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:59.122376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee960d44-5a03-441f-aa4e-012a294faf93 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:59.122647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee960d44-5a03-441f-aa4e-012a294faf93 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:59.123412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee960d44-5a03-441f-aa4e-012a294faf93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:59.124676 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2e11e-3bbb-4fbd-9f7b-292887b3e8ff None admin] RBAC: Authorizing `identity:create_grant(project_id=d9fbb046f448445799c63c2b7d78bd93, user_id=3bd06221895d4193813d7412e1e6a315, role_id=5c2125e2125d40bca6590d50afd677d1)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:59.126156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2e11e-3bbb-4fbd-9f7b-292887b3e8ff None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:59.129379 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee960d44-5a03-441f-aa4e-012a294faf93 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:59.156170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 67/140] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:39:58 2022] PUT /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93/users/3bd06221895d4193813d7412e1e6a315/roles/5c2125e2125d40bca6590d50afd677d1 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:59.160769 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 74/141] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:39:58 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:39:59.728681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6776aa-ba32-4687-baeb-259e42a45a80 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:59.728859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6776aa-ba32-4687-baeb-259e42a45a80 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:59.729168 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6776aa-ba32-4687-baeb-259e42a45a80 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:59.730540 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 68/142] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:39:59.738788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18542c-0f4f-49e7-bb02-39c597c1b5a4 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:59.739163 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18542c-0f4f-49e7-bb02-39c597c1b5a4 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:59.739428 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18542c-0f4f-49e7-bb02-39c597c1b5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:59.763970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-bd18542c-0f4f-49e7-bb02-39c597c1b5a4 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:59.765197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-bd18542c-0f4f-49e7-bb02-39c597c1b5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:59.861178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 75/143] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:39:59.908375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6776aa-ba32-4687-baeb-259e42a45a80 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:00.060895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6776aa-ba32-4687-baeb-259e42a45a80 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:00.104230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac98348-4fa5-4cb0-a5c5-0dbd030001c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:00.105052 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac98348-4fa5-4cb0-a5c5-0dbd030001c7 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:00.105341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac98348-4fa5-4cb0-a5c5-0dbd030001c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:00.105624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac98348-4fa5-4cb0-a5c5-0dbd030001c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:00.117809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac98348-4fa5-4cb0-a5c5-0dbd030001c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-6ac98348-4fa5-4cb0-a5c5-0dbd030001c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:00.129715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:00.130618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:00.131417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:00.131417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:00.131417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:00.131417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:00.131417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:00.131417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:00.131417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:00.131417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:00.131417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:00.131417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:00.131417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:00.131417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:00.132003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 69/144] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:39:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:00.139633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18542c-0f4f-49e7-bb02-39c597c1b5a4 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:00.185798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:00.281760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18542c-0f4f-49e7-bb02-39c597c1b5a4 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:00.323731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9bc80-b0fd-4433-8cec-3b0e3fa10a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:00.324328 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9bc80-b0fd-4433-8cec-3b0e3fa10a35 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:00.324616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9bc80-b0fd-4433-8cec-3b0e3fa10a35 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:00.324895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9bc80-b0fd-4433-8cec-3b0e3fa10a35 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:00.325663 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9bc80-b0fd-4433-8cec-3b0e3fa10a35 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:00.325973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9bc80-b0fd-4433-8cec-3b0e3fa10a35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:00.327397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9bc80-b0fd-4433-8cec-3b0e3fa10a35 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:00.338586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 76/145] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:00.344151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac98348-4fa5-4cb0-a5c5-0dbd030001c7 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:00.400014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:00.404497 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff5421-77c5-45ec-80cf-6f8cd485ea74 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:00.404782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff5421-77c5-45ec-80cf-6f8cd485ea74 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:00.405079 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff5421-77c5-45ec-80cf-6f8cd485ea74 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:00.406464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 77/146] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:00.418785 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ccff7-2b60-4c2c-a4ee-786b30c35115 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:00.419100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ccff7-2b60-4c2c-a4ee-786b30c35115 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:00.419392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ccff7-2b60-4c2c-a4ee-786b30c35115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:00.453490 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-107ccff7-2b60-4c2c-a4ee-786b30c35115 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:00.454909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-107ccff7-2b60-4c2c-a4ee-786b30c35115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:00.492500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac98348-4fa5-4cb0-a5c5-0dbd030001c7 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:00.538701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4717e50-134b-4077-8ce5-af7be759d8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:00.539615 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4717e50-134b-4077-8ce5-af7be759d8f2 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:00.539899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4717e50-134b-4077-8ce5-af7be759d8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:00.540173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4717e50-134b-4077-8ce5-af7be759d8f2 None admin] PATH_INFO: `/v3/users/ff4f688d55724f64b648dda8b1fa5f46` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:00.541460 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4717e50-134b-4077-8ce5-af7be759d8f2 None admin] RBAC: Authorizing `identity:get_user(user_id=ff4f688d55724f64b648dda8b1fa5f46)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:00.543625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4717e50-134b-4077-8ce5-af7be759d8f2 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:00.546904 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 70/147] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:00 2022] GET /identity/v3/users/ff4f688d55724f64b648dda8b1fa5f46 => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:00.553016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4717e50-134b-4077-8ce5-af7be759d8f2 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:00.589522 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 78/148] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:00.632543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ccff7-2b60-4c2c-a4ee-786b30c35115 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:00.663232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:00.746553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:00.757684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4717e50-134b-4077-8ce5-af7be759d8f2 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:00.777759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ccff7-2b60-4c2c-a4ee-786b30c35115 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:00.836047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b5878-4c4c-4765-9ede-a38a0f1b7392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:00.836962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b5878-4c4c-4765-9ede-a38a0f1b7392 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:00.837256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b5878-4c4c-4765-9ede-a38a0f1b7392 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:00.837867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b5878-4c4c-4765-9ede-a38a0f1b7392 None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:00.839417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b5878-4c4c-4765-9ede-a38a0f1b7392 None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:00.841080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b5878-4c4c-4765-9ede-a38a0f1b7392 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:00.843450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 71/149] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:00 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:00.849784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b5878-4c4c-4765-9ede-a38a0f1b7392 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:00.854003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035ef78-5ac9-47f6-939e-afe7c1cc187a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:00.855107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035ef78-5ac9-47f6-939e-afe7c1cc187a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:00.855603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035ef78-5ac9-47f6-939e-afe7c1cc187a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:00.856072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035ef78-5ac9-47f6-939e-afe7c1cc187a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:00.872791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035ef78-5ac9-47f6-939e-afe7c1cc187a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-7035ef78-5ac9-47f6-939e-afe7c1cc187a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:00.881812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:00.882672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:00.883559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:00.883559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:00.883559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:00.883559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:00.883559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:00.883559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:00.883559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:00.883559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:00.883559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:00.883559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:00.883559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:00.883559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:00.886086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 79/150] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:40:00.891752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035ef78-5ac9-47f6-939e-afe7c1cc187a None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:00.988783 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b5878-4c4c-4765-9ede-a38a0f1b7392 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:00.992057 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:01.014751 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035ef78-5ac9-47f6-939e-afe7c1cc187a None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:01.037043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fea19d-6bae-4931-a4d4-1f8457d31fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:01.038507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fea19d-6bae-4931-a4d4-1f8457d31fde None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:01.038507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fea19d-6bae-4931-a4d4-1f8457d31fde None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:01.038698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fea19d-6bae-4931-a4d4-1f8457d31fde None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93/users/ff4f688d55724f64b648dda8b1fa5f46/roles/33252f8dd8084b9ea3fcaf16f4820401` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:01.040727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fea19d-6bae-4931-a4d4-1f8457d31fde None admin] RBAC: Authorizing `identity:create_grant(project_id=d9fbb046f448445799c63c2b7d78bd93, user_id=ff4f688d55724f64b648dda8b1fa5f46, role_id=33252f8dd8084b9ea3fcaf16f4820401)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:01.042173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fea19d-6bae-4931-a4d4-1f8457d31fde None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:01.060829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2e6e8-28a1-4d23-97f5-cbe453cceedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:01.062191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2e6e8-28a1-4d23-97f5-cbe453cceedf None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:01.062726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2e6e8-28a1-4d23-97f5-cbe453cceedf None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:01.063295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2e6e8-28a1-4d23-97f5-cbe453cceedf None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:01.064689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2e6e8-28a1-4d23-97f5-cbe453cceedf None admin] RBAC: Adding query filter params (name=member) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:01.065210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2e6e8-28a1-4d23-97f5-cbe453cceedf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:01.067628 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2e6e8-28a1-4d23-97f5-cbe453cceedf None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:01.070598 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 72/151] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:40:00 2022] PUT /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93/users/ff4f688d55724f64b648dda8b1fa5f46/roles/33252f8dd8084b9ea3fcaf16f4820401 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:40:01.086316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 80/152] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:01.093125 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fea19d-6bae-4931-a4d4-1f8457d31fde None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:01.135922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:01.230690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fea19d-6bae-4931-a4d4-1f8457d31fde None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:01.277826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039305c-a307-4696-ad8d-42559b61932d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:01.278767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039305c-a307-4696-ad8d-42559b61932d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:01.279139 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039305c-a307-4696-ad8d-42559b61932d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:01.279440 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039305c-a307-4696-ad8d-42559b61932d None admin] PATH_INFO: `/v3/users/3bd06221895d4193813d7412e1e6a315` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:01.280796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039305c-a307-4696-ad8d-42559b61932d None admin] RBAC: Authorizing `identity:get_user(user_id=3bd06221895d4193813d7412e1e6a315)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:01.282351 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039305c-a307-4696-ad8d-42559b61932d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:01.284120 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 73/153] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:01 2022] GET /identity/v3/users/3bd06221895d4193813d7412e1e6a315 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:01.290503 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2e6e8-28a1-4d23-97f5-cbe453cceedf None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:01.438062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2e6e8-28a1-4d23-97f5-cbe453cceedf None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:01.486398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa811372-2b59-4a6d-8872-b8942a322055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:01.487092 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa811372-2b59-4a6d-8872-b8942a322055 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:01.487377 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa811372-2b59-4a6d-8872-b8942a322055 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:01.487652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa811372-2b59-4a6d-8872-b8942a322055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:01.502754 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:01.577957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa811372-2b59-4a6d-8872-b8942a322055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:01.579523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa811372-2b59-4a6d-8872-b8942a322055 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:01.698446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 81/154] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:01.709329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039305c-a307-4696-ad8d-42559b61932d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:01.886032 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:01.901043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039305c-a307-4696-ad8d-42559b61932d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:01.942900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58813504-2058-4f9f-b803-5a0c1bb3b6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:01.943945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58813504-2058-4f9f-b803-5a0c1bb3b6a7 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:01.944290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58813504-2058-4f9f-b803-5a0c1bb3b6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:01.944572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58813504-2058-4f9f-b803-5a0c1bb3b6a7 None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:01.951791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58813504-2058-4f9f-b803-5a0c1bb3b6a7 None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:01.953202 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58813504-2058-4f9f-b803-5a0c1bb3b6a7 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:01.955133 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 74/155] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:01 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:01.963312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa811372-2b59-4a6d-8872-b8942a322055 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:02.019088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:02.052908 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde3eac-9900-47df-8899-ce06cf6f7223 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:02.053404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde3eac-9900-47df-8899-ce06cf6f7223 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:02.053476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde3eac-9900-47df-8899-ce06cf6f7223 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:02.054808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 75/156] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:02.060117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf15c4-b41e-4cb5-98cc-f482f5a32d09 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:02.060404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf15c4-b41e-4cb5-98cc-f482f5a32d09 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:02.060682 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf15c4-b41e-4cb5-98cc-f482f5a32d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:02.105464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-bddf15c4-b41e-4cb5-98cc-f482f5a32d09 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:02.106714 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-bddf15c4-b41e-4cb5-98cc-f482f5a32d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:02.135682 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa811372-2b59-4a6d-8872-b8942a322055 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:02.174494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7720ff-942f-4dda-8c44-a254e68026f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:02.175308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7720ff-942f-4dda-8c44-a254e68026f9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:02.175562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7720ff-942f-4dda-8c44-a254e68026f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:02.175838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7720ff-942f-4dda-8c44-a254e68026f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:02.177191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7720ff-942f-4dda-8c44-a254e68026f9 None admin] RBAC: Adding query filter params (role.id=a77e3ea4b46b4a3aaca8e814f53809b5, scope.project.id=d9fbb046f448445799c63c2b7d78bd93, user.id=3bd06221895d4193813d7412e1e6a315) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:02.177572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7720ff-942f-4dda-8c44-a254e68026f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:02.179050 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7720ff-942f-4dda-8c44-a254e68026f9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:02.187701 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 82/157] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:01 2022] GET /identity/v3/role_assignments?user.id=3bd06221895d4193813d7412e1e6a315&scope.project.id=d9fbb046f448445799c63c2b7d78bd93&role.id=a77e3ea4b46b4a3aaca8e814f53809b5 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:02.261235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 76/158] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:40:02.320421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7720ff-942f-4dda-8c44-a254e68026f9 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:02.439832 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7720ff-942f-4dda-8c44-a254e68026f9 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:02.448023 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:02.479899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16892a0-257c-4183-bf24-7416bc7a6fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:02.480705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16892a0-257c-4183-bf24-7416bc7a6fcc None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:02.480985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16892a0-257c-4183-bf24-7416bc7a6fcc None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:02.481252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16892a0-257c-4183-bf24-7416bc7a6fcc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:02.492517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16892a0-257c-4183-bf24-7416bc7a6fcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-d16892a0-257c-4183-bf24-7416bc7a6fcc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:02.500400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:02.501273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:02.502081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:02.502081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:02.502081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:02.502081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:02.502081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:02.502081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:02.502081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:02.502081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:02.502081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:02.502081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:02.502081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:02.502081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:02.502081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 83/159] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:40:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:02.512146 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf15c4-b41e-4cb5-98cc-f482f5a32d09 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:02.555598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1799ffe-9e95-4933-b597-09fb9aca9311 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:02.555887 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1799ffe-9e95-4933-b597-09fb9aca9311 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:02.556147 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1799ffe-9e95-4933-b597-09fb9aca9311 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:02.557939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 84/160] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:02.562996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e584fd-4975-48ae-87ee-2e296e5db7b6 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:02.563289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e584fd-4975-48ae-87ee-2e296e5db7b6 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:02.563574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e584fd-4975-48ae-87ee-2e296e5db7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:02.589130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-75e584fd-4975-48ae-87ee-2e296e5db7b6 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:02.590320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-75e584fd-4975-48ae-87ee-2e296e5db7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:02.703961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:02.710525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 85/161] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:02.722686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf15c4-b41e-4cb5-98cc-f482f5a32d09 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:02.760373 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e584fd-4975-48ae-87ee-2e296e5db7b6 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:02.788267 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89275d2a-d603-42d1-aebb-297ffcf1f0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:02.788871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89275d2a-d603-42d1-aebb-297ffcf1f0ec None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:02.789152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89275d2a-d603-42d1-aebb-297ffcf1f0ec None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:02.789429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89275d2a-d603-42d1-aebb-297ffcf1f0ec None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:02.790185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89275d2a-d603-42d1-aebb-297ffcf1f0ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:02.790491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89275d2a-d603-42d1-aebb-297ffcf1f0ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:02.791901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89275d2a-d603-42d1-aebb-297ffcf1f0ec None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:02.810430 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 77/162] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:02.825598 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89275d2a-d603-42d1-aebb-297ffcf1f0ec None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:02.947986 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:02.956515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e584fd-4975-48ae-87ee-2e296e5db7b6 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:03.021708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc6382-9f44-45d7-84f4-1ab7eafadb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:03.022513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc6382-9f44-45d7-84f4-1ab7eafadb6b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:03.022797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc6382-9f44-45d7-84f4-1ab7eafadb6b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:03.023100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc6382-9f44-45d7-84f4-1ab7eafadb6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:03.055940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc6382-9f44-45d7-84f4-1ab7eafadb6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-9ccc6382-9f44-45d7-84f4-1ab7eafadb6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:03.068306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:03.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:03.072308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:03.072308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:03.072308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:03.072308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:03.072308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:03.072308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:03.072308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:03.072308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:03.072308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:03.072308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:03.072308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:03.072308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:03.072308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 86/163] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:40:03.078526 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89275d2a-d603-42d1-aebb-297ffcf1f0ec None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:03.079135 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc6382-9f44-45d7-84f4-1ab7eafadb6b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:03.156498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363ead2-ecbc-4db8-a654-54ea9da154e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:03.157395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363ead2-ecbc-4db8-a654-54ea9da154e4 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:03.157662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363ead2-ecbc-4db8-a654-54ea9da154e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:03.157932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363ead2-ecbc-4db8-a654-54ea9da154e4 None admin] PATH_INFO: `/v3/users/ea5484f975734ad68c3decb7b2d0c713` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:03.159172 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363ead2-ecbc-4db8-a654-54ea9da154e4 None admin] RBAC: Authorizing `identity:get_user(user_id=ea5484f975734ad68c3decb7b2d0c713)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:03.160582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363ead2-ecbc-4db8-a654-54ea9da154e4 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:03.162284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 78/164] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:02 2022] GET /identity/v3/users/ea5484f975734ad68c3decb7b2d0c713 => generated 247 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:03.172893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363ead2-ecbc-4db8-a654-54ea9da154e4 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:03.217547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:03.274954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc6382-9f44-45d7-84f4-1ab7eafadb6b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:03.337027 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34cc46b-ad5e-460d-9817-86b71432e845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:03.337620 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cc46b-ad5e-460d-9817-86b71432e845 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:03.337901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cc46b-ad5e-460d-9817-86b71432e845 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:03.338171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cc46b-ad5e-460d-9817-86b71432e845 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:03.338951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34cc46b-ad5e-460d-9817-86b71432e845 None admin] RBAC: Adding query filter params (name=member) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:03.339258 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34cc46b-ad5e-460d-9817-86b71432e845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:03.340637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34cc46b-ad5e-460d-9817-86b71432e845 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:03.358458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 87/165] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:03.365526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34cc46b-ad5e-460d-9817-86b71432e845 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:03.394231 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363ead2-ecbc-4db8-a654-54ea9da154e4 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:03.405708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:03.454253 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310e110-5911-41af-8690-fe27818a842f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:03.455086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310e110-5911-41af-8690-fe27818a842f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:03.455378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310e110-5911-41af-8690-fe27818a842f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:03.455660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310e110-5911-41af-8690-fe27818a842f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:03.554455 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34cc46b-ad5e-460d-9817-86b71432e845 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:03.564603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310e110-5911-41af-8690-fe27818a842f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:03.566072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310e110-5911-41af-8690-fe27818a842f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:03.608848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:03.627189 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079982a-0ba9-4079-b9d5-51e566683c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:03.628378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079982a-0ba9-4079-b9d5-51e566683c96 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:03.628864 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079982a-0ba9-4079-b9d5-51e566683c96 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:03.629311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079982a-0ba9-4079-b9d5-51e566683c96 None admin] PATH_INFO: `/v3/users/3bd06221895d4193813d7412e1e6a315` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:03.630942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079982a-0ba9-4079-b9d5-51e566683c96 None admin] RBAC: Authorizing `identity:get_user(user_id=3bd06221895d4193813d7412e1e6a315)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:03.632617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079982a-0ba9-4079-b9d5-51e566683c96 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:03.634774 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 88/166] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:03 2022] GET /identity/v3/users/3bd06221895d4193813d7412e1e6a315 => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:03.641128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079982a-0ba9-4079-b9d5-51e566683c96 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:03.777785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 79/167] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:03.783632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310e110-5911-41af-8690-fe27818a842f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:03.815032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:03.857945 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079982a-0ba9-4079-b9d5-51e566683c96 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:03.959767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdcf86-487b-4cad-b58a-8737eed31214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:03.960923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdcf86-487b-4cad-b58a-8737eed31214 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:03.961426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdcf86-487b-4cad-b58a-8737eed31214 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:03.961897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdcf86-487b-4cad-b58a-8737eed31214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:03.994678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310e110-5911-41af-8690-fe27818a842f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:04.040744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:04.070307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3e845-53d7-4f99-b65e-ce2f3b29886d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:04.071213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3e845-53d7-4f99-b65e-ce2f3b29886d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:04.071503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3e845-53d7-4f99-b65e-ce2f3b29886d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:04.071784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3e845-53d7-4f99-b65e-ce2f3b29886d None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:04.072994 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3e845-53d7-4f99-b65e-ce2f3b29886d None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:04.074454 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3e845-53d7-4f99-b65e-ce2f3b29886d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:04.082454 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 80/168] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:03 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:04.084544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a3e47-563f-4902-a4b6-62447afe9911 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:04.084834 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a3e47-563f-4902-a4b6-62447afe9911 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:04.085112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a3e47-563f-4902-a4b6-62447afe9911 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:04.086749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 81/169] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:04.087479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a3e47-563f-4902-a4b6-62447afe9911 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:04.142344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdcf86-487b-4cad-b58a-8737eed31214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:04.144511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdcf86-487b-4cad-b58a-8737eed31214 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:04.336925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:04.341463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a3e47-563f-4902-a4b6-62447afe9911 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:04.360469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 89/170] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:40:04.361776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdab5c-732d-49d6-b151-41bbf6926d9b None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:04.362063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdab5c-732d-49d6-b151-41bbf6926d9b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:04.362339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdab5c-732d-49d6-b151-41bbf6926d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:04.391724 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-3bcdab5c-732d-49d6-b151-41bbf6926d9b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:04.392979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3bcdab5c-732d-49d6-b151-41bbf6926d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:04.423782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea8ba7-11a0-4c80-b391-453cc07b700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:04.424474 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea8ba7-11a0-4c80-b391-453cc07b700a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:04.424758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea8ba7-11a0-4c80-b391-453cc07b700a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:04.425036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea8ba7-11a0-4c80-b391-453cc07b700a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:04.425897 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea8ba7-11a0-4c80-b391-453cc07b700a None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=d9fbb046f448445799c63c2b7d78bd93, user.id=ea5484f975734ad68c3decb7b2d0c713) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:04.426212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea8ba7-11a0-4c80-b391-453cc07b700a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:04.428315 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea8ba7-11a0-4c80-b391-453cc07b700a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:04.443884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 82/171] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:04 2022] GET /identity/v3/role_assignments?user.id=ea5484f975734ad68c3decb7b2d0c713&scope.project.id=d9fbb046f448445799c63c2b7d78bd93&role.id=0893bb944dd94634accd3906aa391cd2 => generated 608 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:04.444907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea8ba7-11a0-4c80-b391-453cc07b700a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:04.519911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 90/172] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:04.554210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdab5c-732d-49d6-b151-41bbf6926d9b None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:04.582190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:04.712228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdab5c-732d-49d6-b151-41bbf6926d9b None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:04.736399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:04.742793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea8ba7-11a0-4c80-b391-453cc07b700a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:04.768278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0070afa-aff2-422c-bcb0-b1452cd053e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:04.769099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0070afa-aff2-422c-bcb0-b1452cd053e1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:04.769381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0070afa-aff2-422c-bcb0-b1452cd053e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:04.769652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0070afa-aff2-422c-bcb0-b1452cd053e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:04.791760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0070afa-aff2-422c-bcb0-b1452cd053e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-a0070afa-aff2-422c-bcb0-b1452cd053e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:04.799852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:04.800720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:04.801523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:04.801523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:04.801523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:04.801523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:04.801523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:04.801523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:04.801523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:04.801523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:04.801523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:04.801523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:04.801523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:04.801523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:04.801523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 91/173] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:40:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:40:04.802707 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf521b-cdbe-4761-9915-6c1beb33ed73 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:04.803040 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf521b-cdbe-4761-9915-6c1beb33ed73 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:04.803341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf521b-cdbe-4761-9915-6c1beb33ed73 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:04.804555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 92/174] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:04.815014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe162fde-d546-421c-bf5b-26e07c1d0aea None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:04.815304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe162fde-d546-421c-bf5b-26e07c1d0aea None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:04.816632 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe162fde-d546-421c-bf5b-26e07c1d0aea None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:04.817466 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 93/175] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:40:04.818149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe162fde-d546-421c-bf5b-26e07c1d0aea None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:04.828788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a88a6b-5cef-40a4-9639-7100bbd35255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:04.829675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a88a6b-5cef-40a4-9639-7100bbd35255 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:04.829957 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a88a6b-5cef-40a4-9639-7100bbd35255 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:04.830231 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a88a6b-5cef-40a4-9639-7100bbd35255 None admin] PATH_INFO: `/v3/projects/0bd9c3e5e8d34da582e7695992f2b206` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:04.833482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a88a6b-5cef-40a4-9639-7100bbd35255 None admin] RBAC: Authorizing `identity:get_project(project_id=0bd9c3e5e8d34da582e7695992f2b206)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:04.835725 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a88a6b-5cef-40a4-9639-7100bbd35255 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:04.837153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 83/176] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:04 2022] GET /identity/v3/projects/0bd9c3e5e8d34da582e7695992f2b206 => generated 312 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:04.839735 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d5f59-0df5-4f9c-9f06-51132926f33e None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:04.839735 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d5f59-0df5-4f9c-9f06-51132926f33e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:04.839735 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d5f59-0df5-4f9c-9f06-51132926f33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:04.866012 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-982d5f59-0df5-4f9c-9f06-51132926f33e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:04.867273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-982d5f59-0df5-4f9c-9f06-51132926f33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:04.955459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 84/177] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:04.955836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe162fde-d546-421c-bf5b-26e07c1d0aea None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:04.956650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291dfd5-5dcd-4320-8586-c961dc7f08f1 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:04.957002 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291dfd5-5dcd-4320-8586-c961dc7f08f1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:04.957288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291dfd5-5dcd-4320-8586-c961dc7f08f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:04.979592 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-1291dfd5-5dcd-4320-8586-c961dc7f08f1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:04.980702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1291dfd5-5dcd-4320-8586-c961dc7f08f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:05.003159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7893936-1b5c-44e6-9942-ae993503fa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:05.004308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7893936-1b5c-44e6-9942-ae993503fa3a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:05.005353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7893936-1b5c-44e6-9942-ae993503fa3a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:05.005885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7893936-1b5c-44e6-9942-ae993503fa3a None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:05.007070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7893936-1b5c-44e6-9942-ae993503fa3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:05.007621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7893936-1b5c-44e6-9942-ae993503fa3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:05.009378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7893936-1b5c-44e6-9942-ae993503fa3a None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:05.019644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 94/178] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:05.021563 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7893936-1b5c-44e6-9942-ae993503fa3a None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:05.045021 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:05.071604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:05.072824 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 85/179] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:05.073843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291dfd5-5dcd-4320-8586-c961dc7f08f1 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:05.154440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7893936-1b5c-44e6-9942-ae993503fa3a None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:05.191991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215e5a5-cc7b-4195-ab33-b90af8d62615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:05.192855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215e5a5-cc7b-4195-ab33-b90af8d62615 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:05.193148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215e5a5-cc7b-4195-ab33-b90af8d62615 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:05.193430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215e5a5-cc7b-4195-ab33-b90af8d62615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:05.194270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215e5a5-cc7b-4195-ab33-b90af8d62615 None admin] RBAC: Adding query filter params (role.id=a77e3ea4b46b4a3aaca8e814f53809b5, scope.project.id=0bd9c3e5e8d34da582e7695992f2b206, user.id=3bd06221895d4193813d7412e1e6a315) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:05.194586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215e5a5-cc7b-4195-ab33-b90af8d62615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:05.196078 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215e5a5-cc7b-4195-ab33-b90af8d62615 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:05.202202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 95/180] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:05 2022] GET /identity/v3/role_assignments?user.id=3bd06221895d4193813d7412e1e6a315&scope.project.id=0bd9c3e5e8d34da582e7695992f2b206&role.id=a77e3ea4b46b4a3aaca8e814f53809b5 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:05.203044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215e5a5-cc7b-4195-ab33-b90af8d62615 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:05.224175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291dfd5-5dcd-4320-8586-c961dc7f08f1 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:05.264895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430f5d6-b2f4-46b0-a93e-0c9a02b0844d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:05.265740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430f5d6-b2f4-46b0-a93e-0c9a02b0844d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:05.266027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430f5d6-b2f4-46b0-a93e-0c9a02b0844d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:05.266302 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430f5d6-b2f4-46b0-a93e-0c9a02b0844d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:05.279600 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430f5d6-b2f4-46b0-a93e-0c9a02b0844d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-2430f5d6-b2f4-46b0-a93e-0c9a02b0844d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:05.288542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:05.289498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:05.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:05.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:05.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:05.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:05.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:05.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:05.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:05.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:05.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:05.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:05.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:05.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:05.290301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 86/181] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:40:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:05.292316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430f5d6-b2f4-46b0-a93e-0c9a02b0844d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:05.330426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215e5a5-cc7b-4195-ab33-b90af8d62615 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:05.333897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:05.340502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:05.373977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2a2d4-3f1d-4ab8-bf60-4d2421302ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:05.374939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2a2d4-3f1d-4ab8-bf60-4d2421302ee0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:05.375228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2a2d4-3f1d-4ab8-bf60-4d2421302ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:05.375508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2a2d4-3f1d-4ab8-bf60-4d2421302ee0 None admin] PATH_INFO: `/v3/users/3bd06221895d4193813d7412e1e6a315` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:05.376796 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2a2d4-3f1d-4ab8-bf60-4d2421302ee0 None admin] RBAC: Authorizing `identity:get_user(user_id=3bd06221895d4193813d7412e1e6a315)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:05.378264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2a2d4-3f1d-4ab8-bf60-4d2421302ee0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:05.384463 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 96/182] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:05 2022] GET /identity/v3/users/3bd06221895d4193813d7412e1e6a315 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:05.384463 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85209b-548c-4268-8426-40cee447a5fa None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:05.384463 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85209b-548c-4268-8426-40cee447a5fa None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:05.384463 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85209b-548c-4268-8426-40cee447a5fa None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:05.384463 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 97/183] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:05.384865 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807bfe4-9084-4880-8b8a-2e4a31aa9d6f None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:05.384865 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807bfe4-9084-4880-8b8a-2e4a31aa9d6f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:05.385172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807bfe4-9084-4880-8b8a-2e4a31aa9d6f None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:05.388759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 98/184] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:05.391203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807bfe4-9084-4880-8b8a-2e4a31aa9d6f None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:05.417891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430f5d6-b2f4-46b0-a93e-0c9a02b0844d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:05.457150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9a61d-3486-470c-ba73-2625273442fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:05.458901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9a61d-3486-470c-ba73-2625273442fc None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:05.459222 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9a61d-3486-470c-ba73-2625273442fc None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:05.459894 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9a61d-3486-470c-ba73-2625273442fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:05.475371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9a61d-3486-470c-ba73-2625273442fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-23c9a61d-3486-470c-ba73-2625273442fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:05.483470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:05.484512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:05.485298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:05.485298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:05.485298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:05.485298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:05.485298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:05.485298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:05.485298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:05.485298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:05.485298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:05.485298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:05.485298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:05.485298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:05.485298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 87/185] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:40:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:05.486154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9a61d-3486-470c-ba73-2625273442fc None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:05.512547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807bfe4-9084-4880-8b8a-2e4a31aa9d6f None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:05.546758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e057e9d-424f-4dc4-935c-f2f8425f9bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:05.547376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e057e9d-424f-4dc4-935c-f2f8425f9bb7 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:05.547658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e057e9d-424f-4dc4-935c-f2f8425f9bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:05.547937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e057e9d-424f-4dc4-935c-f2f8425f9bb7 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:05.548678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e057e9d-424f-4dc4-935c-f2f8425f9bb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:05.548984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e057e9d-424f-4dc4-935c-f2f8425f9bb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:05.550359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e057e9d-424f-4dc4-935c-f2f8425f9bb7 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:05.558286 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 99/186] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:05.559658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9379687-7b99-4ea0-9039-fca9d836ec4e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:05.559951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9379687-7b99-4ea0-9039-fca9d836ec4e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:05.560229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9379687-7b99-4ea0-9039-fca9d836ec4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:05.576358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f9379687-7b99-4ea0-9039-fca9d836ec4e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:05.577537 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f9379687-7b99-4ea0-9039-fca9d836ec4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:05.588289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9a61d-3486-470c-ba73-2625273442fc None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:05.589928 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:05.604039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:05.623042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb2f62-ac8a-41d5-a7e3-1ba6d1361821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:05.623696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb2f62-ac8a-41d5-a7e3-1ba6d1361821 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:05.623985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb2f62-ac8a-41d5-a7e3-1ba6d1361821 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:05.624268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb2f62-ac8a-41d5-a7e3-1ba6d1361821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:05.647740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 100/187] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:05.649019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb4306-5a69-4b8f-a6f2-8f91cf94f130 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:05.649313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb4306-5a69-4b8f-a6f2-8f91cf94f130 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:05.649599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb4306-5a69-4b8f-a6f2-8f91cf94f130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:05.665607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e1cb4306-5a69-4b8f-a6f2-8f91cf94f130 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:05.666790 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e1cb4306-5a69-4b8f-a6f2-8f91cf94f130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:05.697064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb2f62-ac8a-41d5-a7e3-1ba6d1361821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:05.698552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb2f62-ac8a-41d5-a7e3-1ba6d1361821 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:05.734204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 101/188] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:05.735267 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb4306-5a69-4b8f-a6f2-8f91cf94f130 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:05.805674 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 88/189] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:05.806136 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb2f62-ac8a-41d5-a7e3-1ba6d1361821 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:05.808432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:05.828117 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:05.837505 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb4306-5a69-4b8f-a6f2-8f91cf94f130 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:05.872380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ae3ee-8ee0-4551-a537-a0775e2d502f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:05.873048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ae3ee-8ee0-4551-a537-a0775e2d502f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:05.873342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ae3ee-8ee0-4551-a537-a0775e2d502f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:05.873610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ae3ee-8ee0-4551-a537-a0775e2d502f None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:05.874363 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ae3ee-8ee0-4551-a537-a0775e2d502f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:05.874671 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ae3ee-8ee0-4551-a537-a0775e2d502f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:05.876096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ae3ee-8ee0-4551-a537-a0775e2d502f None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:05.884570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 102/190] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:05.885447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ae3ee-8ee0-4551-a537-a0775e2d502f None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:05.909023 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb2f62-ac8a-41d5-a7e3-1ba6d1361821 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:05.943492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9dbe6b-13a9-4d55-b301-0fdcbc873582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:05.944367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dbe6b-13a9-4d55-b301-0fdcbc873582 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:05.944652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dbe6b-13a9-4d55-b301-0fdcbc873582 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:05.944927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dbe6b-13a9-4d55-b301-0fdcbc873582 None admin] PATH_INFO: `/v3/users/5b75e4fb4b0d4dfe9dfed16707109920` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:05.964149 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9dbe6b-13a9-4d55-b301-0fdcbc873582 None admin] RBAC: Authorizing `identity:get_user(user_id=5b75e4fb4b0d4dfe9dfed16707109920)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:05.965583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9dbe6b-13a9-4d55-b301-0fdcbc873582 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:05.967304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 89/191] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:05 2022] GET /identity/v3/users/5b75e4fb4b0d4dfe9dfed16707109920 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:05.971742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9dbe6b-13a9-4d55-b301-0fdcbc873582 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:05.985647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ae3ee-8ee0-4551-a537-a0775e2d502f None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:06.018338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43347486-85f4-4616-8b00-7b8913b94c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:06.019152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43347486-85f4-4616-8b00-7b8913b94c96 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:06.019436 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43347486-85f4-4616-8b00-7b8913b94c96 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:06.019713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43347486-85f4-4616-8b00-7b8913b94c96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:06.021444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43347486-85f4-4616-8b00-7b8913b94c96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:06.022930 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43347486-85f4-4616-8b00-7b8913b94c96 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:06.024535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 103/192] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:06.025452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43347486-85f4-4616-8b00-7b8913b94c96 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:06.033368 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:06.049485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:06.076186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9dbe6b-13a9-4d55-b301-0fdcbc873582 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:06.112022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845a7fb-6735-40f5-8394-c14f65bc5f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:06.112823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845a7fb-6735-40f5-8394-c14f65bc5f48 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:06.113108 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845a7fb-6735-40f5-8394-c14f65bc5f48 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:06.113389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845a7fb-6735-40f5-8394-c14f65bc5f48 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:06.124296 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845a7fb-6735-40f5-8394-c14f65bc5f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:06.124417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43347486-85f4-4616-8b00-7b8913b94c96 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-a845a7fb-6735-40f5-8394-c14f65bc5f48 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:06.132648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:06.133718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:06.134554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:06.134554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:06.134554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:06.134554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:06.134554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:06.134554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:06.134554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:06.134554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:06.134554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:06.134554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:06.134554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:06.134554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:06.134554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 90/193] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:06.135658 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845a7fb-6735-40f5-8394-c14f65bc5f48 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:06.159878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1fc97-a300-4f1c-8c7a-c9dbfd00edf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:06.161393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1fc97-a300-4f1c-8c7a-c9dbfd00edf7 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:06.161795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1fc97-a300-4f1c-8c7a-c9dbfd00edf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:06.162074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1fc97-a300-4f1c-8c7a-c9dbfd00edf7 None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:06.163308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1fc97-a300-4f1c-8c7a-c9dbfd00edf7 None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:06.164761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1fc97-a300-4f1c-8c7a-c9dbfd00edf7 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:06.166437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 104/194] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:06 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:06.167219 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1fc97-a300-4f1c-8c7a-c9dbfd00edf7 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:06.247024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845a7fb-6735-40f5-8394-c14f65bc5f48 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:06.259445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:06.266973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:06.270744 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1fc97-a300-4f1c-8c7a-c9dbfd00edf7 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:06.282317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698a5e9-43fa-44ce-ae6d-f8fab584f309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:06.283219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698a5e9-43fa-44ce-ae6d-f8fab584f309 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:06.283502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698a5e9-43fa-44ce-ae6d-f8fab584f309 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:06.283780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698a5e9-43fa-44ce-ae6d-f8fab584f309 None admin] PATH_INFO: `/v3/users/5b75e4fb4b0d4dfe9dfed16707109920` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:06.284979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698a5e9-43fa-44ce-ae6d-f8fab584f309 None admin] RBAC: Authorizing `identity:get_user(user_id=5b75e4fb4b0d4dfe9dfed16707109920)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:06.287140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698a5e9-43fa-44ce-ae6d-f8fab584f309 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:06.288788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 91/195] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:06 2022] GET /identity/v3/users/5b75e4fb4b0d4dfe9dfed16707109920 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:06.290611 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698a5e9-43fa-44ce-ae6d-f8fab584f309 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:06.305485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d78f8-5067-4d71-8380-43ccb26f98fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:06.306137 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d78f8-5067-4d71-8380-43ccb26f98fc None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:06.306423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d78f8-5067-4d71-8380-43ccb26f98fc None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:06.306706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d78f8-5067-4d71-8380-43ccb26f98fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:06.374314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d78f8-5067-4d71-8380-43ccb26f98fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:06.375907 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d78f8-5067-4d71-8380-43ccb26f98fc None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:06.392112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698a5e9-43fa-44ce-ae6d-f8fab584f309 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:06.426512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb4898-fcee-40e2-a460-67cc89705449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:06.427588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb4898-fcee-40e2-a460-67cc89705449 None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:06.427877 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb4898-fcee-40e2-a460-67cc89705449 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:06.428157 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb4898-fcee-40e2-a460-67cc89705449 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:06.428864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb4898-fcee-40e2-a460-67cc89705449 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:06.430342 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb4898-fcee-40e2-a460-67cc89705449 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:06.456606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 92/196] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:40:06 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:40:06.457430 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb4898-fcee-40e2-a460-67cc89705449 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:06.478170 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:06.490721 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 105/197] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:06.491701 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d78f8-5067-4d71-8380-43ccb26f98fc None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:06.517754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:06.584338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb4898-fcee-40e2-a460-67cc89705449 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:06.610950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d78f8-5067-4d71-8380-43ccb26f98fc None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:06.623594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755701fd-0f22-45d4-87b1-979f0f1ad1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:06.624281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755701fd-0f22-45d4-87b1-979f0f1ad1d7 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:06.624626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755701fd-0f22-45d4-87b1-979f0f1ad1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:06.624846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755701fd-0f22-45d4-87b1-979f0f1ad1d7 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:06.625609 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755701fd-0f22-45d4-87b1-979f0f1ad1d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:06.625921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755701fd-0f22-45d4-87b1-979f0f1ad1d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:06.627365 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755701fd-0f22-45d4-87b1-979f0f1ad1d7 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:06.637756 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 93/198] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:06.639810 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755701fd-0f22-45d4-87b1-979f0f1ad1d7 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:06.656812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beadca4-a6a1-4cb6-8ad1-690bf66bb80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:06.657850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beadca4-a6a1-4cb6-8ad1-690bf66bb80c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:06.657850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beadca4-a6a1-4cb6-8ad1-690bf66bb80c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:06.657850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beadca4-a6a1-4cb6-8ad1-690bf66bb80c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:06.659199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beadca4-a6a1-4cb6-8ad1-690bf66bb80c None admin] RBAC: Adding query filter params (role.id=5c2125e2125d40bca6590d50afd677d1, scope.project.id=d9fbb046f448445799c63c2b7d78bd93, user.id=3bd06221895d4193813d7412e1e6a315) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:06.659199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beadca4-a6a1-4cb6-8ad1-690bf66bb80c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:06.660454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beadca4-a6a1-4cb6-8ad1-690bf66bb80c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:06.669700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 106/199] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:06 2022] GET /identity/v3/role_assignments?user.id=3bd06221895d4193813d7412e1e6a315&scope.project.id=d9fbb046f448445799c63c2b7d78bd93&role.id=5c2125e2125d40bca6590d50afd677d1 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:06.670598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beadca4-a6a1-4cb6-8ad1-690bf66bb80c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:06.745755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:06.775077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755701fd-0f22-45d4-87b1-979f0f1ad1d7 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:06.785754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:06.812427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beadca4-a6a1-4cb6-8ad1-690bf66bb80c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:06.828787 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71dcaed-44a6-4fd9-b4d6-02abbc255cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:06.830261 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71dcaed-44a6-4fd9-b4d6-02abbc255cf2 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:06.830730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71dcaed-44a6-4fd9-b4d6-02abbc255cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:06.831438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71dcaed-44a6-4fd9-b4d6-02abbc255cf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:06.855814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d37e8-ff7c-47ac-aabd-0cb55c35a1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:06.856799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d37e8-ff7c-47ac-aabd-0cb55c35a1fd None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:06.857086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d37e8-ff7c-47ac-aabd-0cb55c35a1fd None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:06.857368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d37e8-ff7c-47ac-aabd-0cb55c35a1fd None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:06.858641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d37e8-ff7c-47ac-aabd-0cb55c35a1fd None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:06.860199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d37e8-ff7c-47ac-aabd-0cb55c35a1fd None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:06.862523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 107/200] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:06 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:06.863588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d37e8-ff7c-47ac-aabd-0cb55c35a1fd None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:06.916542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71dcaed-44a6-4fd9-b4d6-02abbc255cf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:06.917915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71dcaed-44a6-4fd9-b4d6-02abbc255cf2 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:06.996946 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d37e8-ff7c-47ac-aabd-0cb55c35a1fd None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:07.026803 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:07.042736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5192d7e-e274-48f4-9769-ef1a3ae61a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:07.043637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5192d7e-e274-48f4-9769-ef1a3ae61a1b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:07.043929 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5192d7e-e274-48f4-9769-ef1a3ae61a1b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:07.044210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5192d7e-e274-48f4-9769-ef1a3ae61a1b None admin] PATH_INFO: `/v3/users/ff4f688d55724f64b648dda8b1fa5f46` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:07.045007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 94/201] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:07.045555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5192d7e-e274-48f4-9769-ef1a3ae61a1b None admin] RBAC: Authorizing `identity:get_user(user_id=ff4f688d55724f64b648dda8b1fa5f46)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:07.045865 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71dcaed-44a6-4fd9-b4d6-02abbc255cf2 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:07.047144 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5192d7e-e274-48f4-9769-ef1a3ae61a1b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:07.048861 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 108/202] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:06 2022] GET /identity/v3/users/ff4f688d55724f64b648dda8b1fa5f46 => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:07.053207 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5192d7e-e274-48f4-9769-ef1a3ae61a1b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:07.063216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:07.171056 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71dcaed-44a6-4fd9-b4d6-02abbc255cf2 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:07.190879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5192d7e-e274-48f4-9769-ef1a3ae61a1b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:07.212309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bb0bb-a5fe-49b4-9cd7-f42561f198e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:07.213022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bb0bb-a5fe-49b4-9cd7-f42561f198e6 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:07.213363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bb0bb-a5fe-49b4-9cd7-f42561f198e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:07.213643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bb0bb-a5fe-49b4-9cd7-f42561f198e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:07.214665 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bb0bb-a5fe-49b4-9cd7-f42561f198e6 None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea, user.id=5b75e4fb4b0d4dfe9dfed16707109920) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:07.215223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bb0bb-a5fe-49b4-9cd7-f42561f198e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:07.216578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bb0bb-a5fe-49b4-9cd7-f42561f198e6 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:07.223874 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 95/203] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:07 2022] GET /identity/v3/role_assignments?user.id=5b75e4fb4b0d4dfe9dfed16707109920&scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea&role.id=0893bb944dd94634accd3906aa391cd2 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:07.231085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bb0bb-a5fe-49b4-9cd7-f42561f198e6 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:07.249552 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d7c45-7c40-49b9-aca8-34d5b9aed3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:07.250445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d7c45-7c40-49b9-aca8-34d5b9aed3e1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:07.250729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d7c45-7c40-49b9-aca8-34d5b9aed3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:07.251051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d7c45-7c40-49b9-aca8-34d5b9aed3e1 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:07.255916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d7c45-7c40-49b9-aca8-34d5b9aed3e1 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:07.257410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d7c45-7c40-49b9-aca8-34d5b9aed3e1 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:07.259859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 109/204] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:07 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:07.266616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d7c45-7c40-49b9-aca8-34d5b9aed3e1 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:07.371790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:07.415713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bb0bb-a5fe-49b4-9cd7-f42561f198e6 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:07.416736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:07.483811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8381f-12df-4856-957c-18a395ecf7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:07.484394 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8381f-12df-4856-957c-18a395ecf7e7 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:07.484681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8381f-12df-4856-957c-18a395ecf7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:07.484962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8381f-12df-4856-957c-18a395ecf7e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:07.513125 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d7c45-7c40-49b9-aca8-34d5b9aed3e1 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:07.597717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8381f-12df-4856-957c-18a395ecf7e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:07.600053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8381f-12df-4856-957c-18a395ecf7e7 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:07.608175 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1fd8a-4ad5-4158-87a0-c11d167000b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:07.608948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1fd8a-4ad5-4158-87a0-c11d167000b1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:07.609237 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1fd8a-4ad5-4158-87a0-c11d167000b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:07.609513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1fd8a-4ad5-4158-87a0-c11d167000b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:07.610367 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1fd8a-4ad5-4158-87a0-c11d167000b1 None admin] RBAC: Adding query filter params (role.id=5c2125e2125d40bca6590d50afd677d1, scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea, user.id=5b75e4fb4b0d4dfe9dfed16707109920) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:07.610684 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1fd8a-4ad5-4158-87a0-c11d167000b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:07.612158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1fd8a-4ad5-4158-87a0-c11d167000b1 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:07.628171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 110/205] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:07 2022] GET /identity/v3/role_assignments?user.id=5b75e4fb4b0d4dfe9dfed16707109920&scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea&role.id=5c2125e2125d40bca6590d50afd677d1 => generated 259 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:07.695837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:07.758544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 96/206] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:07.763922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1fd8a-4ad5-4158-87a0-c11d167000b1 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:07.919235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1fd8a-4ad5-4158-87a0-c11d167000b1 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:07.944387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:07.956991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08fc74-720b-4956-8ec0-533743dfed63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:07.959036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08fc74-720b-4956-8ec0-533743dfed63 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:07.959578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08fc74-720b-4956-8ec0-533743dfed63 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:07.960057 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08fc74-720b-4956-8ec0-533743dfed63 None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:07.961487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08fc74-720b-4956-8ec0-533743dfed63 None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:07.963179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08fc74-720b-4956-8ec0-533743dfed63 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:07.965529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 111/207] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:07 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:07.971533 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8381f-12df-4856-957c-18a395ecf7e7 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:08.126668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8381f-12df-4856-957c-18a395ecf7e7 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:08.176261 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b8307-683d-4f96-87b4-6a54ede547a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:08.177436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b8307-683d-4f96-87b4-6a54ede547a9 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:08.177722 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b8307-683d-4f96-87b4-6a54ede547a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:08.177987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b8307-683d-4f96-87b4-6a54ede547a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:08.178833 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b8307-683d-4f96-87b4-6a54ede547a9 None admin] RBAC: Adding query filter params (role.id=33252f8dd8084b9ea3fcaf16f4820401, scope.project.id=d9fbb046f448445799c63c2b7d78bd93, user.id=ff4f688d55724f64b648dda8b1fa5f46) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:08.179173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b8307-683d-4f96-87b4-6a54ede547a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:08.180604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b8307-683d-4f96-87b4-6a54ede547a9 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:08.192016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 97/208] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:07 2022] GET /identity/v3/role_assignments?user.id=ff4f688d55724f64b648dda8b1fa5f46&scope.project.id=d9fbb046f448445799c63c2b7d78bd93&role.id=33252f8dd8084b9ea3fcaf16f4820401 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:09.211688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace0930-91f5-4b15-9398-062291c30a8e None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:09.212707 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace0930-91f5-4b15-9398-062291c30a8e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:09.213224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace0930-91f5-4b15-9398-062291c30a8e None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:09.215126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 112/209] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:09.220966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67201ce-c6f0-4327-b77d-0120e1bc3ab6 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:09.221805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67201ce-c6f0-4327-b77d-0120e1bc3ab6 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:09.222309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67201ce-c6f0-4327-b77d-0120e1bc3ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:09.242782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-d67201ce-c6f0-4327-b77d-0120e1bc3ab6 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:09.244455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d67201ce-c6f0-4327-b77d-0120e1bc3ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:09.276022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:09.325815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 98/210] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:09.346096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace0930-91f5-4b15-9398-062291c30a8e None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:09.428008 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5322e4-c945-4e20-b47f-d1a63ff16fd3 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:09.428626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5322e4-c945-4e20-b47f-d1a63ff16fd3 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:09.429142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5322e4-c945-4e20-b47f-d1a63ff16fd3 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:09.430985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 99/211] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:09.436110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3a474-33b0-4c08-811e-f39d7c7119a1 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:09.436621 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3a474-33b0-4c08-811e-f39d7c7119a1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:09.437104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3a474-33b0-4c08-811e-f39d7c7119a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:09.455800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace0930-91f5-4b15-9398-062291c30a8e None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:09.459487 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-80b3a474-33b0-4c08-811e-f39d7c7119a1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:09.461026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-80b3a474-33b0-4c08-811e-f39d7c7119a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:09.505569 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af60489-1eeb-4ae4-8688-b6b13de79559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:09.506696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af60489-1eeb-4ae4-8688-b6b13de79559 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:09.507232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af60489-1eeb-4ae4-8688-b6b13de79559 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:09.507703 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af60489-1eeb-4ae4-8688-b6b13de79559 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:09.519256 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af60489-1eeb-4ae4-8688-b6b13de79559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-8af60489-1eeb-4ae4-8688-b6b13de79559 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:09.527916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:09.528786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:09.529544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:09.529544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:09.529544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:09.529544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:09.529544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:09.529544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:09.529544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:09.529544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:09.529544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:09.529544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:09.529544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:09.529544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:09.532195 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 113/212] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:09.538738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af60489-1eeb-4ae4-8688-b6b13de79559 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:09.551022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 100/213] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:40:09.571346 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3a474-33b0-4c08-811e-f39d7c7119a1 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:09.601816 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:09.671708 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:09.672255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af60489-1eeb-4ae4-8688-b6b13de79559 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:09.681192 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3a474-33b0-4c08-811e-f39d7c7119a1 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:09.709810 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291215e-bddc-4ef0-9132-7034d88ca4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:09.710752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291215e-bddc-4ef0-9132-7034d88ca4cb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:09.711067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291215e-bddc-4ef0-9132-7034d88ca4cb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:09.711350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291215e-bddc-4ef0-9132-7034d88ca4cb None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:09.712707 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291215e-bddc-4ef0-9132-7034d88ca4cb None admin] RBAC: Adding query filter params (name=member) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:09.713040 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291215e-bddc-4ef0-9132-7034d88ca4cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:09.714752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291215e-bddc-4ef0-9132-7034d88ca4cb None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:09.717336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4fd46-6bb9-4a93-90ab-83d4cae7a410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:09.718118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4fd46-6bb9-4a93-90ab-83d4cae7a410 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:09.718399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4fd46-6bb9-4a93-90ab-83d4cae7a410 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:09.718672 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4fd46-6bb9-4a93-90ab-83d4cae7a410 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:09.719878 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4fd46-6bb9-4a93-90ab-83d4cae7a410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:09.721318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4fd46-6bb9-4a93-90ab-83d4cae7a410 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:09.723243 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 101/214] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:09.724335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528241b-0e52-4f91-b241-15df9cbe0bb2 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:09.724622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528241b-0e52-4f91-b241-15df9cbe0bb2 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:09.724897 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528241b-0e52-4f91-b241-15df9cbe0bb2 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:09.726283 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 102/215] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:09.732156 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 114/216] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:09.732318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528241b-0e52-4f91-b241-15df9cbe0bb2 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:09.732414 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ea7c6-f381-44f1-ba16-1556f29dd13b None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:09.732541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ea7c6-f381-44f1-ba16-1556f29dd13b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:09.732829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ea7c6-f381-44f1-ba16-1556f29dd13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:09.755247 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-b49ea7c6-f381-44f1-ba16-1556f29dd13b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:09.757019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-b49ea7c6-f381-44f1-ba16-1556f29dd13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:09.829143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 115/217] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:09.829985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ea7c6-f381-44f1-ba16-1556f29dd13b None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:09.857097 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528241b-0e52-4f91-b241-15df9cbe0bb2 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:09.873735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:09.900737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9e7fb-f97e-4b55-b3c5-4106ae820416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:09.901405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e7fb-f97e-4b55-b3c5-4106ae820416 None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:09.901687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e7fb-f97e-4b55-b3c5-4106ae820416 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:09.901958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e7fb-f97e-4b55-b3c5-4106ae820416 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:09.902656 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9e7fb-f97e-4b55-b3c5-4106ae820416 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:09.904140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9e7fb-f97e-4b55-b3c5-4106ae820416 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:09.935241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 103/218] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:40:09 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:40:09.936060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9e7fb-f97e-4b55-b3c5-4106ae820416 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:09.937751 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ea7c6-f381-44f1-ba16-1556f29dd13b None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:09.981043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:09.996765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f5802-c9a1-41b8-8bf2-8e22fc43e95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:09.997683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f5802-c9a1-41b8-8bf2-8e22fc43e95e None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:09.997964 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f5802-c9a1-41b8-8bf2-8e22fc43e95e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:09.998235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f5802-c9a1-41b8-8bf2-8e22fc43e95e None admin] PATH_INFO: `/v3/users/598969ce89a04164ade8384e458f4c4b` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:10.027352 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f5802-c9a1-41b8-8bf2-8e22fc43e95e None admin] RBAC: Authorizing `identity:get_user(user_id=598969ce89a04164ade8384e458f4c4b)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:10.029865 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f5802-c9a1-41b8-8bf2-8e22fc43e95e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:10.033121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 116/219] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:09 2022] GET /identity/v3/users/598969ce89a04164ade8384e458f4c4b => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:10.038706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f5802-c9a1-41b8-8bf2-8e22fc43e95e None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:10.120910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9e7fb-f97e-4b55-b3c5-4106ae820416 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:10.164338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:10.165774 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f4e5a-f3c5-4ff3-96ef-50fd571c5e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:10.166680 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f4e5a-f3c5-4ff3-96ef-50fd571c5e62 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:10.167087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f4e5a-f3c5-4ff3-96ef-50fd571c5e62 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:10.167433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f4e5a-f3c5-4ff3-96ef-50fd571c5e62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:10.169611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f5802-c9a1-41b8-8bf2-8e22fc43e95e None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:10.180692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f4e5a-f3c5-4ff3-96ef-50fd571c5e62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-101f4e5a-f3c5-4ff3-96ef-50fd571c5e62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:10.188732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:10.189652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:10.190445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:10.190445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:10.190445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:10.190445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:10.190445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:10.190445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:10.190445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:10.190445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:10.190445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:10.190445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:10.190445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:10.190445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:10.190445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 104/220] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:40:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:10.191650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97367a89-1da0-4c35-89bc-b5699b471447 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:10.191863 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97367a89-1da0-4c35-89bc-b5699b471447 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:10.192104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97367a89-1da0-4c35-89bc-b5699b471447 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:10.193397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 105/221] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:10.197704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97367a89-1da0-4c35-89bc-b5699b471447 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:10.210133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb0589-86c5-4881-8d27-191114ab9404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:10.210907 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb0589-86c5-4881-8d27-191114ab9404 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:10.211138 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb0589-86c5-4881-8d27-191114ab9404 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:10.211424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb0589-86c5-4881-8d27-191114ab9404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:10.301897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb0589-86c5-4881-8d27-191114ab9404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:10.303327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb0589-86c5-4881-8d27-191114ab9404 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:10.303817 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:10.337000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97367a89-1da0-4c35-89bc-b5699b471447 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:10.382321 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4fd68-4c43-43bd-bc58-2e3a0d0bd777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:10.383034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4fd68-4c43-43bd-bc58-2e3a0d0bd777 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:10.383581 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4fd68-4c43-43bd-bc58-2e3a0d0bd777 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:10.383865 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4fd68-4c43-43bd-bc58-2e3a0d0bd777 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:10.384635 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4fd68-4c43-43bd-bc58-2e3a0d0bd777 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:10.384938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4fd68-4c43-43bd-bc58-2e3a0d0bd777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:10.386604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4fd68-4c43-43bd-bc58-2e3a0d0bd777 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:10.398003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 106/222] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:10.403037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5551a67-c8fa-4fab-a948-4a2afec6fff0 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:10.403037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5551a67-c8fa-4fab-a948-4a2afec6fff0 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:10.403037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5551a67-c8fa-4fab-a948-4a2afec6fff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:10.421701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a5551a67-c8fa-4fab-a948-4a2afec6fff0 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:10.423384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a5551a67-c8fa-4fab-a948-4a2afec6fff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:10.430200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:10.438957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 117/223] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:40:10.438957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb0589-86c5-4881-8d27-191114ab9404 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:10.504364 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 107/224] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:10.505711 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5551a67-c8fa-4fab-a948-4a2afec6fff0 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:10.573375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb0589-86c5-4881-8d27-191114ab9404 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:10.584678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:10.617555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfe957-9b7a-439a-aaab-4d8c455b002f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:10.618758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfe957-9b7a-439a-aaab-4d8c455b002f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:10.619075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfe957-9b7a-439a-aaab-4d8c455b002f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:10.619387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfe957-9b7a-439a-aaab-4d8c455b002f None admin] PATH_INFO: `/v3/users/5b75e4fb4b0d4dfe9dfed16707109920` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:10.620856 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfe957-9b7a-439a-aaab-4d8c455b002f None admin] RBAC: Authorizing `identity:get_user(user_id=5b75e4fb4b0d4dfe9dfed16707109920)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:10.622076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5551a67-c8fa-4fab-a948-4a2afec6fff0 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:10.622694 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfe957-9b7a-439a-aaab-4d8c455b002f None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:10.624671 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 118/225] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:10 2022] GET /identity/v3/users/5b75e4fb4b0d4dfe9dfed16707109920 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:10.629250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfe957-9b7a-439a-aaab-4d8c455b002f None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:10.666132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1cbd3-7b00-4a6d-a8fc-a88237dc0280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:10.667483 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1cbd3-7b00-4a6d-a8fc-a88237dc0280 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:10.668129 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1cbd3-7b00-4a6d-a8fc-a88237dc0280 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:10.668597 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1cbd3-7b00-4a6d-a8fc-a88237dc0280 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:10.669978 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1cbd3-7b00-4a6d-a8fc-a88237dc0280 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:10.671723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1cbd3-7b00-4a6d-a8fc-a88237dc0280 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:10.673849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 108/226] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:10 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:10.679837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1cbd3-7b00-4a6d-a8fc-a88237dc0280 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:10.706612 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:10.747767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfe957-9b7a-439a-aaab-4d8c455b002f None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:10.787252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b864f4-eed6-4ab0-9863-c00e0e3fc8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:10.788567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b864f4-eed6-4ab0-9863-c00e0e3fc8fb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:10.789045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b864f4-eed6-4ab0-9863-c00e0e3fc8fb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:10.789512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b864f4-eed6-4ab0-9863-c00e0e3fc8fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:10.801233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b864f4-eed6-4ab0-9863-c00e0e3fc8fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:10.809224 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1cbd3-7b00-4a6d-a8fc-a88237dc0280 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-12b864f4-eed6-4ab0-9863-c00e0e3fc8fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:10.810534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:10.811425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:10.812204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:10.812204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:10.812204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:10.812204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:10.812204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:10.812204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:10.812204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:10.812204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:10.812204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:10.812204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:10.812204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:10.812204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:10.815196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 119/227] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:40:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:10.816482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b864f4-eed6-4ab0-9863-c00e0e3fc8fb None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:10.855666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3f19c-34cb-42a9-9def-dc394433c3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:10.856485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3f19c-34cb-42a9-9def-dc394433c3a8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:10.857648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3f19c-34cb-42a9-9def-dc394433c3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:10.857940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3f19c-34cb-42a9-9def-dc394433c3a8 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:10.859380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3f19c-34cb-42a9-9def-dc394433c3a8 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:10.860846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3f19c-34cb-42a9-9def-dc394433c3a8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:10.862584 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 109/228] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:10 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:10.864058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3f19c-34cb-42a9-9def-dc394433c3a8 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:10.875218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:10.935808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b864f4-eed6-4ab0-9863-c00e0e3fc8fb None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:10.966615 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:10.976810 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efecfdb-8e01-4b97-b448-14ba428a807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:10.977915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efecfdb-8e01-4b97-b448-14ba428a807f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:10.978395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efecfdb-8e01-4b97-b448-14ba428a807f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:10.978886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efecfdb-8e01-4b97-b448-14ba428a807f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:10.980067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efecfdb-8e01-4b97-b448-14ba428a807f None admin] RBAC: Adding query filter params (role.id=a77e3ea4b46b4a3aaca8e814f53809b5, scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea, user.id=598969ce89a04164ade8384e458f4c4b) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:10.980613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efecfdb-8e01-4b97-b448-14ba428a807f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:10.982303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efecfdb-8e01-4b97-b448-14ba428a807f None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:10.987173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3f19c-34cb-42a9-9def-dc394433c3a8 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:10.989404 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 120/229] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:10 2022] GET /identity/v3/role_assignments?user.id=598969ce89a04164ade8384e458f4c4b&scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea&role.id=a77e3ea4b46b4a3aaca8e814f53809b5 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:10.995253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efecfdb-8e01-4b97-b448-14ba428a807f None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:11.024094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1a6f6-6b7e-41ac-9c59-d4713781a85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:11.024775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1a6f6-6b7e-41ac-9c59-d4713781a85e None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:11.030901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1a6f6-6b7e-41ac-9c59-d4713781a85e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:11.030901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1a6f6-6b7e-41ac-9c59-d4713781a85e None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:11.030901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf1a6f6-6b7e-41ac-9c59-d4713781a85e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:11.030901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf1a6f6-6b7e-41ac-9c59-d4713781a85e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:11.030901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf1a6f6-6b7e-41ac-9c59-d4713781a85e None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:11.036833 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 110/230] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:11.047038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1a6f6-6b7e-41ac-9c59-d4713781a85e None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:11.143154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efecfdb-8e01-4b97-b448-14ba428a807f None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:11.182298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:11.183787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df247154-c0fb-4440-bd18-f3e111f0b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:11.184944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df247154-c0fb-4440-bd18-f3e111f0b048 None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:11.185429 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df247154-c0fb-4440-bd18-f3e111f0b048 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:11.185920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df247154-c0fb-4440-bd18-f3e111f0b048 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea/users/5b75e4fb4b0d4dfe9dfed16707109920/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:11.188543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df247154-c0fb-4440-bd18-f3e111f0b048 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea, user_id=5b75e4fb4b0d4dfe9dfed16707109920, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:11.190244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df247154-c0fb-4440-bd18-f3e111f0b048 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:11.208655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 121/231] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:40:10 2022] PUT /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea/users/5b75e4fb4b0d4dfe9dfed16707109920/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:40:11.239813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1a6f6-6b7e-41ac-9c59-d4713781a85e None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:11.294265 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab145a96-b7b8-451a-8938-2ec80f169db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:11.295641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab145a96-b7b8-451a-8938-2ec80f169db0 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:11.296140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab145a96-b7b8-451a-8938-2ec80f169db0 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:11.296624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab145a96-b7b8-451a-8938-2ec80f169db0 None admin] PATH_INFO: `/v3/users/5b75e4fb4b0d4dfe9dfed16707109920` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:11.298385 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab145a96-b7b8-451a-8938-2ec80f169db0 None admin] RBAC: Authorizing `identity:get_user(user_id=5b75e4fb4b0d4dfe9dfed16707109920)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:11.300298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab145a96-b7b8-451a-8938-2ec80f169db0 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:11.303065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 111/232] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:11 2022] GET /identity/v3/users/5b75e4fb4b0d4dfe9dfed16707109920 => generated 283 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:11.308545 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df247154-c0fb-4440-bd18-f3e111f0b048 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:11.372184 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:11.433142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df247154-c0fb-4440-bd18-f3e111f0b048 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:11.470669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00e8a6-6c65-4d1e-8ee6-83d48d7d9188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:11.471584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00e8a6-6c65-4d1e-8ee6-83d48d7d9188 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:11.471863 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00e8a6-6c65-4d1e-8ee6-83d48d7d9188 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:11.472139 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00e8a6-6c65-4d1e-8ee6-83d48d7d9188 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:11.473923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00e8a6-6c65-4d1e-8ee6-83d48d7d9188 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:11.475473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00e8a6-6c65-4d1e-8ee6-83d48d7d9188 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:11.477506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 122/233] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:11 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:11.483927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab145a96-b7b8-451a-8938-2ec80f169db0 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:11.600142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab145a96-b7b8-451a-8938-2ec80f169db0 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:11.636250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd9268-1e4f-467c-8aa9-6f31424512a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:11.637199 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd9268-1e4f-467c-8aa9-6f31424512a8 None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:11.637490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd9268-1e4f-467c-8aa9-6f31424512a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:11.637770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd9268-1e4f-467c-8aa9-6f31424512a8 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea/users/5b75e4fb4b0d4dfe9dfed16707109920/roles/5c2125e2125d40bca6590d50afd677d1` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:11.640118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd9268-1e4f-467c-8aa9-6f31424512a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea, user_id=5b75e4fb4b0d4dfe9dfed16707109920, role_id=5c2125e2125d40bca6590d50afd677d1)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:11.641618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd9268-1e4f-467c-8aa9-6f31424512a8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:11.646897 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:11.660344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 112/234] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:40:11 2022] PUT /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea/users/5b75e4fb4b0d4dfe9dfed16707109920/roles/5c2125e2125d40bca6590d50afd677d1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:40:12.522176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ee3cc-d8fb-4db8-b395-46ff2cd2ac9b None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:12.522689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ee3cc-d8fb-4db8-b395-46ff2cd2ac9b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:12.522689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ee3cc-d8fb-4db8-b395-46ff2cd2ac9b None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:12.524242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 123/235] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:40:12.529740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffc427-ef14-41c5-a589-3ab93665d80c None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:12.530182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffc427-ef14-41c5-a589-3ab93665d80c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:12.530298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffc427-ef14-41c5-a589-3ab93665d80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:12.551060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-deffc427-ef14-41c5-a589-3ab93665d80c None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:12.552347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-deffc427-ef14-41c5-a589-3ab93665d80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:12.600690 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e737b-8164-43a0-a24d-83713068f972 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:12.600991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e737b-8164-43a0-a24d-83713068f972 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:12.601246 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e737b-8164-43a0-a24d-83713068f972 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:12.602735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 124/236] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:40:12.607937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a75f9-b829-43ec-b78f-334dd317fbe1 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:12.608453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a75f9-b829-43ec-b78f-334dd317fbe1 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:12.608995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a75f9-b829-43ec-b78f-334dd317fbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:12.631637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-9e4a75f9-b829-43ec-b78f-334dd317fbe1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:12.633257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9e4a75f9-b829-43ec-b78f-334dd317fbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:12.670990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 113/237] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:40:12.696168 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffc427-ef14-41c5-a589-3ab93665d80c None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:12.715304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 125/238] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:40:12.740868 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a75f9-b829-43ec-b78f-334dd317fbe1 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:12.807103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:12.854011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a75f9-b829-43ec-b78f-334dd317fbe1 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:12.893034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f767c5-b400-4bb1-94e8-82abc2cb65d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:12.894056 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f767c5-b400-4bb1-94e8-82abc2cb65d9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:12.894346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f767c5-b400-4bb1-94e8-82abc2cb65d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:12.894624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f767c5-b400-4bb1-94e8-82abc2cb65d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:12.897548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f767c5-b400-4bb1-94e8-82abc2cb65d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:12.899171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f767c5-b400-4bb1-94e8-82abc2cb65d9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:12.902343 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 126/239] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:12.914277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f767c5-b400-4bb1-94e8-82abc2cb65d9 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:12.924862 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffc427-ef14-41c5-a589-3ab93665d80c None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:12.955228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:12.966787 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afdb79-90e4-4a49-be07-37919590cf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:12.967907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afdb79-90e4-4a49-be07-37919590cf93 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:12.968411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afdb79-90e4-4a49-be07-37919590cf93 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:12.968889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afdb79-90e4-4a49-be07-37919590cf93 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:12.992518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afdb79-90e4-4a49-be07-37919590cf93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-88afdb79-90e4-4a49-be07-37919590cf93 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:13.001952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:13.002798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:13.003632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:13.003632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:13.003632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:13.003632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:13.003632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:13.003632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:13.003632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:13.003632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:13.003632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:13.003632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:13.003632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:13.003632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:13.013825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 114/240] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:13.019788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afdb79-90e4-4a49-be07-37919590cf93 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:13.110579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f767c5-b400-4bb1-94e8-82abc2cb65d9 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:13.126131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afdb79-90e4-4a49-be07-37919590cf93 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:13.144260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:13.162886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a194b-e238-4bc8-ab1f-eefc54f5e83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:13.162886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a194b-e238-4bc8-ab1f-eefc54f5e83b None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:13.162886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a194b-e238-4bc8-ab1f-eefc54f5e83b None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:13.162886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a194b-e238-4bc8-ab1f-eefc54f5e83b None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:13.163280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a194b-e238-4bc8-ab1f-eefc54f5e83b None admin] RBAC: Adding query filter params (name=reader) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:13.163280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a194b-e238-4bc8-ab1f-eefc54f5e83b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:13.164438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a194b-e238-4bc8-ab1f-eefc54f5e83b None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:13.172907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 115/241] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:13.178609 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a194b-e238-4bc8-ab1f-eefc54f5e83b None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:13.189160 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30de369-a7c0-4fc8-92f8-e3dfc18b9ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:13.190320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30de369-a7c0-4fc8-92f8-e3dfc18b9ce7 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:13.190835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30de369-a7c0-4fc8-92f8-e3dfc18b9ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:13.191427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30de369-a7c0-4fc8-92f8-e3dfc18b9ce7 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:13.193645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30de369-a7c0-4fc8-92f8-e3dfc18b9ce7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:13.195853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30de369-a7c0-4fc8-92f8-e3dfc18b9ce7 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:13.230469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:13.250400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 127/242] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:40:12 2022] POST /identity/v3/users => generated 293 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:40:13.291011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a194b-e238-4bc8-ab1f-eefc54f5e83b None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:13.328307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358d65e-f94b-4c32-af06-4b2e0a6de8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:13.329244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358d65e-f94b-4c32-af06-4b2e0a6de8f1 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:13.329532 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358d65e-f94b-4c32-af06-4b2e0a6de8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:13.329809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358d65e-f94b-4c32-af06-4b2e0a6de8f1 None admin] PATH_INFO: `/v3/users/40d171854ae14801a9053fe7f688805d` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:13.351647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358d65e-f94b-4c32-af06-4b2e0a6de8f1 None admin] RBAC: Authorizing `identity:get_user(user_id=40d171854ae14801a9053fe7f688805d)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:13.353143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358d65e-f94b-4c32-af06-4b2e0a6de8f1 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:13.354914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 116/243] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:13 2022] GET /identity/v3/users/40d171854ae14801a9053fe7f688805d => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:13.363544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30de369-a7c0-4fc8-92f8-e3dfc18b9ce7 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:13.500011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30de369-a7c0-4fc8-92f8-e3dfc18b9ce7 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:13.540341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa81cb4-6ff3-4d60-b86d-65a5319b7be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:13.541191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa81cb4-6ff3-4d60-b86d-65a5319b7be3 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:13.541317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa81cb4-6ff3-4d60-b86d-65a5319b7be3 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:13.541561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa81cb4-6ff3-4d60-b86d-65a5319b7be3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:13.591318 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:13.622416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa81cb4-6ff3-4d60-b86d-65a5319b7be3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:13.624035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa81cb4-6ff3-4d60-b86d-65a5319b7be3 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:13.735527 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 128/244] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:13.742647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358d65e-f94b-4c32-af06-4b2e0a6de8f1 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:13.829069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab58a-d563-4e8a-bf39-6b1663966249 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:13.829388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab58a-d563-4e8a-bf39-6b1663966249 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:13.829777 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab58a-d563-4e8a-bf39-6b1663966249 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:13.831566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 129/245] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:13.838220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d48e7-c9b1-4186-adcf-d9e8d52b9caf None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:13.838510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d48e7-c9b1-4186-adcf-d9e8d52b9caf None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:13.838784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d48e7-c9b1-4186-adcf-d9e8d52b9caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:13.856893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358d65e-f94b-4c32-af06-4b2e0a6de8f1 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:13.857192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-630d48e7-c9b1-4186-adcf-d9e8d52b9caf None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:13.858394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-630d48e7-c9b1-4186-adcf-d9e8d52b9caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:13.867220 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:13.901288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c7b02-7e07-4a54-9e0e-0cb4701d5417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:13.902534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c7b02-7e07-4a54-9e0e-0cb4701d5417 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:13.903050 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c7b02-7e07-4a54-9e0e-0cb4701d5417 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:13.903578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c7b02-7e07-4a54-9e0e-0cb4701d5417 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:13.904999 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c7b02-7e07-4a54-9e0e-0cb4701d5417 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:13.906637 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c7b02-7e07-4a54-9e0e-0cb4701d5417 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:13.915487 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 117/246] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:13 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:13.916693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c7b02-7e07-4a54-9e0e-0cb4701d5417 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:13.939584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 130/247] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:13.963695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d48e7-c9b1-4186-adcf-d9e8d52b9caf None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:13.971898 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:14.055250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c7b02-7e07-4a54-9e0e-0cb4701d5417 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:14.080395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d48e7-c9b1-4186-adcf-d9e8d52b9caf None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:14.099415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2ece0-c2b4-45d5-971c-a9cc1410f25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:14.100343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2ece0-c2b4-45d5-971c-a9cc1410f25a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:14.100628 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2ece0-c2b4-45d5-971c-a9cc1410f25a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:14.101159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2ece0-c2b4-45d5-971c-a9cc1410f25a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:14.102400 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2ece0-c2b4-45d5-971c-a9cc1410f25a None admin] RBAC: Adding query filter params (role.id=33252f8dd8084b9ea3fcaf16f4820401, scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea, user.id=40d171854ae14801a9053fe7f688805d) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:14.102713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2ece0-c2b4-45d5-971c-a9cc1410f25a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:14.104181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2ece0-c2b4-45d5-971c-a9cc1410f25a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:14.112414 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 118/248] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:13 2022] GET /identity/v3/role_assignments?user.id=40d171854ae14801a9053fe7f688805d&scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea&role.id=33252f8dd8084b9ea3fcaf16f4820401 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:14.123653 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10500ef7-5d50-41f0-a620-424ef5762bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:14.124548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10500ef7-5d50-41f0-a620-424ef5762bc4 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:14.124834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10500ef7-5d50-41f0-a620-424ef5762bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:14.125181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10500ef7-5d50-41f0-a620-424ef5762bc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:14.139021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10500ef7-5d50-41f0-a620-424ef5762bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-10500ef7-5d50-41f0-a620-424ef5762bc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:14.147709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:14.148669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:14.149450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:14.149450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:14.149450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:14.149450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:14.149450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:14.149450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:14.149450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:14.149450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:14.149450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:14.149450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:14.149450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:14.149450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:14.149450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 131/249] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:40:14.154896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2ece0-c2b4-45d5-971c-a9cc1410f25a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:14.200765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:14.270946 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2ece0-c2b4-45d5-971c-a9cc1410f25a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:14.311496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473901e-a2a0-4aec-af50-c71d5601fd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:14.312181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473901e-a2a0-4aec-af50-c71d5601fd9b None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:14.312471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473901e-a2a0-4aec-af50-c71d5601fd9b None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:14.312748 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473901e-a2a0-4aec-af50-c71d5601fd9b None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:14.313504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473901e-a2a0-4aec-af50-c71d5601fd9b None admin] RBAC: Adding query filter params (name=member) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:14.313812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473901e-a2a0-4aec-af50-c71d5601fd9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:14.315223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473901e-a2a0-4aec-af50-c71d5601fd9b None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:14.325304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 119/250] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:14.330956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10500ef7-5d50-41f0-a620-424ef5762bc4 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:14.411545 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:14.448482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10500ef7-5d50-41f0-a620-424ef5762bc4 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:14.491294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d182c-34ba-41a5-93d0-6ba7843e4817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:14.492164 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d182c-34ba-41a5-93d0-6ba7843e4817 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:14.492388 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a0202-18b4-4ed7-b41d-9314bf7c5ce6 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:14.492492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d182c-34ba-41a5-93d0-6ba7843e4817 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:14.492705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d182c-34ba-41a5-93d0-6ba7843e4817 None admin] PATH_INFO: `/v3/users/598969ce89a04164ade8384e458f4c4b` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:14.493167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a0202-18b4-4ed7-b41d-9314bf7c5ce6 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:14.493705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a0202-18b4-4ed7-b41d-9314bf7c5ce6 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:14.495565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 120/251] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:14.497817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d182c-34ba-41a5-93d0-6ba7843e4817 None admin] RBAC: Authorizing `identity:get_user(user_id=598969ce89a04164ade8384e458f4c4b)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:14.499324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d182c-34ba-41a5-93d0-6ba7843e4817 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:14.501052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 132/252] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:14 2022] GET /identity/v3/users/598969ce89a04164ade8384e458f4c4b => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:14.501705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f387137-f22d-4661-b9e4-02eecd1af047 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:14.502265 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f387137-f22d-4661-b9e4-02eecd1af047 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:14.502739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f387137-f22d-4661-b9e4-02eecd1af047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:14.507309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d182c-34ba-41a5-93d0-6ba7843e4817 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:14.536888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-9f387137-f22d-4661-b9e4-02eecd1af047 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:14.538420 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9f387137-f22d-4661-b9e4-02eecd1af047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:14.637809 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d182c-34ba-41a5-93d0-6ba7843e4817 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:14.674136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b609e-61dc-42cb-ab66-a659058301a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:14.675053 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b609e-61dc-42cb-ab66-a659058301a1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:14.675337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b609e-61dc-42cb-ab66-a659058301a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:14.675611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b609e-61dc-42cb-ab66-a659058301a1 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:14.677530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b609e-61dc-42cb-ab66-a659058301a1 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:14.681319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b609e-61dc-42cb-ab66-a659058301a1 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:14.681464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 133/253] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:14 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:14.682611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6702c-9fb0-49b7-80c7-971dd3e9ebfa None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:14.683173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6702c-9fb0-49b7-80c7-971dd3e9ebfa None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:14.683521 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6702c-9fb0-49b7-80c7-971dd3e9ebfa None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:14.687376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 134/254] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:14.687376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:14.687376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6702c-9fb0-49b7-80c7-971dd3e9ebfa None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:14.726371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 121/255] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:40:14.732183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c04f15-e2a4-435a-9ea7-b9cdc7934ce3 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:14.732684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c04f15-e2a4-435a-9ea7-b9cdc7934ce3 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:14.733167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c04f15-e2a4-435a-9ea7-b9cdc7934ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:14.764655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-f3c04f15-e2a4-435a-9ea7-b9cdc7934ce3 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:14.766534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f3c04f15-e2a4-435a-9ea7-b9cdc7934ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:14.799357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:14.862371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 122/256] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:14.863456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c04f15-e2a4-435a-9ea7-b9cdc7934ce3 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:14.895934 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6702c-9fb0-49b7-80c7-971dd3e9ebfa None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:14.975703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c04f15-e2a4-435a-9ea7-b9cdc7934ce3 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:15.007229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fb57a-94b3-46c6-a53b-e244a0654fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:15.008428 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fb57a-94b3-46c6-a53b-e244a0654fc3 None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:15.008886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fb57a-94b3-46c6-a53b-e244a0654fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:15.009361 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fb57a-94b3-46c6-a53b-e244a0654fc3 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea/users/598969ce89a04164ade8384e458f4c4b/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:15.011285 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf239bb-9108-4aa0-aca2-ad3dc991c53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:15.012178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf239bb-9108-4aa0-aca2-ad3dc991c53e None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:15.012465 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf239bb-9108-4aa0-aca2-ad3dc991c53e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:15.012736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf239bb-9108-4aa0-aca2-ad3dc991c53e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:15.013431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fb57a-94b3-46c6-a53b-e244a0654fc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea, user_id=598969ce89a04164ade8384e458f4c4b, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:15.015043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fb57a-94b3-46c6-a53b-e244a0654fc3 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:15.025185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf239bb-9108-4aa0-aca2-ad3dc991c53e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-daf239bb-9108-4aa0-aca2-ad3dc991c53e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:15.033686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:15.034796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:15.036407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:15.036407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:15.036407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:15.036407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:15.036407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:15.036407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:15.036407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:15.036407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:15.036407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:15.036407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:15.036407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:15.036407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:15.038606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 123/257] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:40:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:15.039726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf239bb-9108-4aa0-aca2-ad3dc991c53e None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:15.054729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 135/258] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:40:14 2022] PUT /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea/users/598969ce89a04164ade8384e458f4c4b/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:40:15.055501 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fb57a-94b3-46c6-a53b-e244a0654fc3 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:15.071297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:15.164215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:15.216220 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf239bb-9108-4aa0-aca2-ad3dc991c53e None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:15.237155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fb57a-94b3-46c6-a53b-e244a0654fc3 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:15.279213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34555e50-761b-4e57-9ba6-869d494e87b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:15.280248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34555e50-761b-4e57-9ba6-869d494e87b9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:15.280914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34555e50-761b-4e57-9ba6-869d494e87b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:15.281387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34555e50-761b-4e57-9ba6-869d494e87b9 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:15.282337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34555e50-761b-4e57-9ba6-869d494e87b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:15.282889 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34555e50-761b-4e57-9ba6-869d494e87b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:15.284471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34555e50-761b-4e57-9ba6-869d494e87b9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:15.287888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda0057-a3df-4952-85ff-93340bcbae2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:15.288718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda0057-a3df-4952-85ff-93340bcbae2b None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:15.289000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda0057-a3df-4952-85ff-93340bcbae2b None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:15.289276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda0057-a3df-4952-85ff-93340bcbae2b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:15.295359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 136/259] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:15.300921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34555e50-761b-4e57-9ba6-869d494e87b9 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:15.315331 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda0057-a3df-4952-85ff-93340bcbae2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-8cda0057-a3df-4952-85ff-93340bcbae2b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:15.332055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:15.333126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:15.333980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:15.333980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:15.333980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:15.333980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:15.333980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:15.333980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:15.333980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:15.333980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:15.333980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:15.333980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:15.333980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:15.333980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:15.333980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 124/260] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:40:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:15.339421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda0057-a3df-4952-85ff-93340bcbae2b None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:15.422193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34555e50-761b-4e57-9ba6-869d494e87b9 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:15.462554 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d3d0e-2d5e-4298-a9cf-8e0eadd36a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:15.463784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d3d0e-2d5e-4298-a9cf-8e0eadd36a85 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:15.464177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d3d0e-2d5e-4298-a9cf-8e0eadd36a85 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:15.464558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d3d0e-2d5e-4298-a9cf-8e0eadd36a85 None admin] PATH_INFO: `/v3/users/5b75e4fb4b0d4dfe9dfed16707109920` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:15.465887 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d3d0e-2d5e-4298-a9cf-8e0eadd36a85 None admin] RBAC: Authorizing `identity:get_user(user_id=5b75e4fb4b0d4dfe9dfed16707109920)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:15.467592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d3d0e-2d5e-4298-a9cf-8e0eadd36a85 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:15.469451 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 137/261] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:15 2022] GET /identity/v3/users/5b75e4fb4b0d4dfe9dfed16707109920 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:15.469868 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:15.474835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d3d0e-2d5e-4298-a9cf-8e0eadd36a85 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:15.541931 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:15.569412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda0057-a3df-4952-85ff-93340bcbae2b None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:15.595395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d3d0e-2d5e-4298-a9cf-8e0eadd36a85 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:15.607528 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bf0d6-ba4b-447f-b4bd-3dccdb99723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:15.608213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bf0d6-ba4b-447f-b4bd-3dccdb99723f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:15.608501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bf0d6-ba4b-447f-b4bd-3dccdb99723f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:15.608783 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bf0d6-ba4b-447f-b4bd-3dccdb99723f None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:15.609533 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bf0d6-ba4b-447f-b4bd-3dccdb99723f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:15.609845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bf0d6-ba4b-447f-b4bd-3dccdb99723f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:15.611256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bf0d6-ba4b-447f-b4bd-3dccdb99723f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:15.620477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 125/262] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:15.626116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bf0d6-ba4b-447f-b4bd-3dccdb99723f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:15.661089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2766c5-1e01-4ae1-a334-7475c86e833b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:15.661776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2766c5-1e01-4ae1-a334-7475c86e833b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:15.662037 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2766c5-1e01-4ae1-a334-7475c86e833b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:15.662313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2766c5-1e01-4ae1-a334-7475c86e833b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:15.766823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bf0d6-ba4b-447f-b4bd-3dccdb99723f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:15.793650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2766c5-1e01-4ae1-a334-7475c86e833b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:15.799150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:15.803189 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2766c5-1e01-4ae1-a334-7475c86e833b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:15.842795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b735e-e800-49b8-90ae-b0910fca1272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:15.843721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b735e-e800-49b8-90ae-b0910fca1272 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:15.844066 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b735e-e800-49b8-90ae-b0910fca1272 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:15.844350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b735e-e800-49b8-90ae-b0910fca1272 None admin] PATH_INFO: `/v3/users/5b75e4fb4b0d4dfe9dfed16707109920` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:15.845618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b735e-e800-49b8-90ae-b0910fca1272 None admin] RBAC: Authorizing `identity:get_user(user_id=5b75e4fb4b0d4dfe9dfed16707109920)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:15.847088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b735e-e800-49b8-90ae-b0910fca1272 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:15.849747 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 126/263] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:15 2022] GET /identity/v3/users/5b75e4fb4b0d4dfe9dfed16707109920 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:15.855086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b735e-e800-49b8-90ae-b0910fca1272 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:15.875584 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:15.969777 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 138/264] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:15.975048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2766c5-1e01-4ae1-a334-7475c86e833b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:15.978113 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b735e-e800-49b8-90ae-b0910fca1272 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:16.033053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e985b-bf4c-4256-b168-6fa1a585bbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:16.033593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e985b-bf4c-4256-b168-6fa1a585bbd8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:16.033883 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e985b-bf4c-4256-b168-6fa1a585bbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:16.034163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e985b-bf4c-4256-b168-6fa1a585bbd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:16.136970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:16.153015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2766c5-1e01-4ae1-a334-7475c86e833b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:16.168989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e985b-bf4c-4256-b168-6fa1a585bbd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:16.170748 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e985b-bf4c-4256-b168-6fa1a585bbd8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:16.187168 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:16.226272 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77831d1-0bd8-41d1-b2d9-b3cc88ea9c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:16.227167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77831d1-0bd8-41d1-b2d9-b3cc88ea9c0b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:16.227452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77831d1-0bd8-41d1-b2d9-b3cc88ea9c0b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:16.227727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77831d1-0bd8-41d1-b2d9-b3cc88ea9c0b None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:16.231981 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77831d1-0bd8-41d1-b2d9-b3cc88ea9c0b None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:16.233427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77831d1-0bd8-41d1-b2d9-b3cc88ea9c0b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:16.235477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 139/265] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:15 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:16.252415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77831d1-0bd8-41d1-b2d9-b3cc88ea9c0b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:16.376372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 127/266] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:40:16.382740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e985b-bf4c-4256-b168-6fa1a585bbd8 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:16.502805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77831d1-0bd8-41d1-b2d9-b3cc88ea9c0b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:16.540933 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedb3d8-bd6f-47a5-88b3-db6373af6e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:16.541602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb3d8-bd6f-47a5-88b3-db6373af6e06 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:16.541886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb3d8-bd6f-47a5-88b3-db6373af6e06 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:16.542159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb3d8-bd6f-47a5-88b3-db6373af6e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:16.544335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedb3d8-bd6f-47a5-88b3-db6373af6e06 None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea, user.id=5b75e4fb4b0d4dfe9dfed16707109920) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:16.544335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedb3d8-bd6f-47a5-88b3-db6373af6e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:16.545047 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedb3d8-bd6f-47a5-88b3-db6373af6e06 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:16.552263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 140/267] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:16 2022] GET /identity/v3/role_assignments?user.id=5b75e4fb4b0d4dfe9dfed16707109920&scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea&role.id=0893bb944dd94634accd3906aa391cd2 => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:16.625027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e985b-bf4c-4256-b168-6fa1a585bbd8 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:16.638285 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:16.675547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102c70b-c0a5-4717-82aa-1e8fc705b846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:16.677301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102c70b-c0a5-4717-82aa-1e8fc705b846 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:16.677785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102c70b-c0a5-4717-82aa-1e8fc705b846 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:16.678282 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102c70b-c0a5-4717-82aa-1e8fc705b846 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:16.679728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102c70b-c0a5-4717-82aa-1e8fc705b846 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:16.681381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102c70b-c0a5-4717-82aa-1e8fc705b846 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:16.683467 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 128/268] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:16 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:16.689584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedb3d8-bd6f-47a5-88b3-db6373af6e06 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:16.739641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:16.829876 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedb3d8-bd6f-47a5-88b3-db6373af6e06 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:16.868307 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd09a9-9363-4d97-8fac-2abda2aa8e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:16.868969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd09a9-9363-4d97-8fac-2abda2aa8e85 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:16.869249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd09a9-9363-4d97-8fac-2abda2aa8e85 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:16.869522 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd09a9-9363-4d97-8fac-2abda2aa8e85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:16.870359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd09a9-9363-4d97-8fac-2abda2aa8e85 None admin] RBAC: Adding query filter params (role.id=5c2125e2125d40bca6590d50afd677d1, scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea, user.id=5b75e4fb4b0d4dfe9dfed16707109920) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:16.870669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd09a9-9363-4d97-8fac-2abda2aa8e85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:16.872390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd09a9-9363-4d97-8fac-2abda2aa8e85 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:16.879105 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 141/269] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:16 2022] GET /identity/v3/role_assignments?user.id=5b75e4fb4b0d4dfe9dfed16707109920&scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea&role.id=5c2125e2125d40bca6590d50afd677d1 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:17.283081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a113038-ee72-4ee8-b0cd-aacc8e555828 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:17.284313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a113038-ee72-4ee8-b0cd-aacc8e555828 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:17.284815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a113038-ee72-4ee8-b0cd-aacc8e555828 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:17.287291 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 129/270] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:17.293869 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35902682-d698-4a9c-bf61-c8aa62fa26c5 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:17.294185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35902682-d698-4a9c-bf61-c8aa62fa26c5 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:17.294411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35902682-d698-4a9c-bf61-c8aa62fa26c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:17.316770 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-35902682-d698-4a9c-bf61-c8aa62fa26c5 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:17.318388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-35902682-d698-4a9c-bf61-c8aa62fa26c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:17.425911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 142/271] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:17.451666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a113038-ee72-4ee8-b0cd-aacc8e555828 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:17.484178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a47b3c-5319-47a3-a7fc-5917eda65bc9 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:17.484455 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a47b3c-5319-47a3-a7fc-5917eda65bc9 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:17.484726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a47b3c-5319-47a3-a7fc-5917eda65bc9 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:17.486405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 143/272] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:40:17.492125 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748ab26-7102-4a22-a51d-02f7cfe467aa None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:17.492418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748ab26-7102-4a22-a51d-02f7cfe467aa None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:17.492702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748ab26-7102-4a22-a51d-02f7cfe467aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:17.510061 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-8748ab26-7102-4a22-a51d-02f7cfe467aa None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:17.511324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8748ab26-7102-4a22-a51d-02f7cfe467aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:17.524003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:17.581386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a113038-ee72-4ee8-b0cd-aacc8e555828 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:17.594883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 144/273] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:40:17.594883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bc50a-4385-421d-944f-9816ae1ad3be None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:17.594883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bc50a-4385-421d-944f-9816ae1ad3be None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:17.594883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bc50a-4385-421d-944f-9816ae1ad3be None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:17.594883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 145/274] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:17.601838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024392eb-4c56-44b1-8d92-d74e409a6d86 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:17.602141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024392eb-4c56-44b1-8d92-d74e409a6d86 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:17.602427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024392eb-4c56-44b1-8d92-d74e409a6d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:17.622152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-024392eb-4c56-44b1-8d92-d74e409a6d86 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:17.623373 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-024392eb-4c56-44b1-8d92-d74e409a6d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:17.627179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3d61a-0b71-4e59-8af2-bb7d0df0849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:17.627989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3d61a-0b71-4e59-8af2-bb7d0df0849a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:17.628405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3d61a-0b71-4e59-8af2-bb7d0df0849a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:17.628686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3d61a-0b71-4e59-8af2-bb7d0df0849a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:17.630199 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3d61a-0b71-4e59-8af2-bb7d0df0849a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:17.631980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3d61a-0b71-4e59-8af2-bb7d0df0849a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:17.635182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 130/275] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:17.636605 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3d61a-0b71-4e59-8af2-bb7d0df0849a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:17.688482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:17.699025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 146/276] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:17.708644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024392eb-4c56-44b1-8d92-d74e409a6d86 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:17.747680 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3d61a-0b71-4e59-8af2-bb7d0df0849a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:17.784751 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb017a5-696d-4761-90a0-8b310334d855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:17.785701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb017a5-696d-4761-90a0-8b310334d855 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:17.785984 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb017a5-696d-4761-90a0-8b310334d855 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:17.786439 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb017a5-696d-4761-90a0-8b310334d855 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:17.795316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:17.797086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb017a5-696d-4761-90a0-8b310334d855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-ddb017a5-696d-4761-90a0-8b310334d855 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:17.804949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:17.806010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:17.807074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:17.807074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:17.807074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:17.807074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:17.807074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:17.807074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:17.807074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:17.807074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:17.807074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:17.807074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:17.807074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:17.807074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:17.811102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 131/277] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:17.812086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb017a5-696d-4761-90a0-8b310334d855 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:17.826729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024392eb-4c56-44b1-8d92-d74e409a6d86 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:17.867199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc878471-a903-4e3a-afd1-8e671c247df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:17.867875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc878471-a903-4e3a-afd1-8e671c247df6 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:17.868159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc878471-a903-4e3a-afd1-8e671c247df6 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:17.868433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc878471-a903-4e3a-afd1-8e671c247df6 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:17.869137 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc878471-a903-4e3a-afd1-8e671c247df6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:17.870917 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc878471-a903-4e3a-afd1-8e671c247df6 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:17.899009 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 147/278] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:40:17 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:40:17.899009 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc878471-a903-4e3a-afd1-8e671c247df6 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:17.944915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb017a5-696d-4761-90a0-8b310334d855 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:17.955730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:17.996569 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f72cf2-887b-486b-ac38-235771397594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:17.997575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f72cf2-887b-486b-ac38-235771397594 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:17.997864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f72cf2-887b-486b-ac38-235771397594 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:17.998143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f72cf2-887b-486b-ac38-235771397594 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:18.014737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f72cf2-887b-486b-ac38-235771397594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:18.020359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc878471-a903-4e3a-afd1-8e671c247df6 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-44f72cf2-887b-486b-ac38-235771397594 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:18.024196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:18.025117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:18.025902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:18.025902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:18.025902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:18.025902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:18.025902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:18.025902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:18.025902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:18.025902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:18.025902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:18.025902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:18.025902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:18.025902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:18.025902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 132/279] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:18.027336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a87bdc-eea1-4cec-9570-5a0576331c93 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:18.027628 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a87bdc-eea1-4cec-9570-5a0576331c93 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:18.027908 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a87bdc-eea1-4cec-9570-5a0576331c93 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:18.029394 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 133/280] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:18.031861 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a87bdc-eea1-4cec-9570-5a0576331c93 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:18.060600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:18.067968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b6f9f-1f5d-4b9e-aca5-8347fd2c0dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:18.068786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6f9f-1f5d-4b9e-aca5-8347fd2c0dc4 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:18.069431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6f9f-1f5d-4b9e-aca5-8347fd2c0dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:18.070150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6f9f-1f5d-4b9e-aca5-8347fd2c0dc4 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:18.071049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b6f9f-1f5d-4b9e-aca5-8347fd2c0dc4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:18.071277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b6f9f-1f5d-4b9e-aca5-8347fd2c0dc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:18.073098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b6f9f-1f5d-4b9e-aca5-8347fd2c0dc4 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:18.088710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 148/281] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:18.088710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300762f-c470-44b4-9e92-5459f3fba584 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:18.088710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300762f-c470-44b4-9e92-5459f3fba584 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:18.088710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300762f-c470-44b4-9e92-5459f3fba584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:18.104958 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-8300762f-c470-44b4-9e92-5459f3fba584 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:18.106171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8300762f-c470-44b4-9e92-5459f3fba584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:18.157611 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a87bdc-eea1-4cec-9570-5a0576331c93 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:18.183597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 149/282] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:18.184461 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300762f-c470-44b4-9e92-5459f3fba584 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:18.202724 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ef060-2f19-452d-a4c5-ad30a655ed48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:18.203422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ef060-2f19-452d-a4c5-ad30a655ed48 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:18.203705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ef060-2f19-452d-a4c5-ad30a655ed48 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:18.203980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ef060-2f19-452d-a4c5-ad30a655ed48 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:18.204725 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ef060-2f19-452d-a4c5-ad30a655ed48 None admin] RBAC: Adding query filter params (name=member) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:18.205026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ef060-2f19-452d-a4c5-ad30a655ed48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:18.206404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ef060-2f19-452d-a4c5-ad30a655ed48 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:18.223961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 134/283] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:18.223961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ef060-2f19-452d-a4c5-ad30a655ed48 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:18.257552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:18.321702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300762f-c470-44b4-9e92-5459f3fba584 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:18.352363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ef060-2f19-452d-a4c5-ad30a655ed48 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:18.353324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:18.367524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69f763-57bb-44a8-8f72-2bd52aca818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:18.368803 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69f763-57bb-44a8-8f72-2bd52aca818d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:18.369194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69f763-57bb-44a8-8f72-2bd52aca818d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:18.369607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69f763-57bb-44a8-8f72-2bd52aca818d None admin] PATH_INFO: `/v3/users/40d171854ae14801a9053fe7f688805d` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:18.371818 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69f763-57bb-44a8-8f72-2bd52aca818d None admin] RBAC: Authorizing `identity:get_user(user_id=40d171854ae14801a9053fe7f688805d)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:18.373974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69f763-57bb-44a8-8f72-2bd52aca818d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:18.376329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 150/284] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:18 2022] GET /identity/v3/users/40d171854ae14801a9053fe7f688805d => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:18.377459 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69f763-57bb-44a8-8f72-2bd52aca818d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:18.412361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf85bd1-08b9-4c97-9f56-916bab766ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:18.413142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf85bd1-08b9-4c97-9f56-916bab766ce8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:18.413433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf85bd1-08b9-4c97-9f56-916bab766ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:18.413701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf85bd1-08b9-4c97-9f56-916bab766ce8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:18.424459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf85bd1-08b9-4c97-9f56-916bab766ce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-7bf85bd1-08b9-4c97-9f56-916bab766ce8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:18.432659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:18.433785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:18.434762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:18.434762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:18.434762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:18.434762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:18.434762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:18.434762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:18.434762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:18.434762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:18.434762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:18.434762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:18.434762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:18.434762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:18.435439 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 135/285] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:18.438378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf85bd1-08b9-4c97-9f56-916bab766ce8 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:18.513387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69f763-57bb-44a8-8f72-2bd52aca818d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:18.549363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:18.549761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334307b-2295-459e-abe9-5045761c1f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:18.550839 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334307b-2295-459e-abe9-5045761c1f6e None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:18.551281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334307b-2295-459e-abe9-5045761c1f6e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:18.551827 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334307b-2295-459e-abe9-5045761c1f6e None admin] PATH_INFO: `/v3/users/19a5838c8dfa415cbb6cd30897f8f54b` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:18.566342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334307b-2295-459e-abe9-5045761c1f6e None admin] RBAC: Authorizing `identity:get_user(user_id=19a5838c8dfa415cbb6cd30897f8f54b)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:18.568646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334307b-2295-459e-abe9-5045761c1f6e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:18.568978 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf85bd1-08b9-4c97-9f56-916bab766ce8 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:18.575080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 151/286] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:18 2022] GET /identity/v3/users/19a5838c8dfa415cbb6cd30897f8f54b => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:18.575080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334307b-2295-459e-abe9-5045761c1f6e None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:18.611505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d793f6-ae13-4919-8e6e-4b06f29be1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:18.612429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d793f6-ae13-4919-8e6e-4b06f29be1b3 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:18.612719 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d793f6-ae13-4919-8e6e-4b06f29be1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:18.612992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d793f6-ae13-4919-8e6e-4b06f29be1b3 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:18.614663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d793f6-ae13-4919-8e6e-4b06f29be1b3 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:18.616197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d793f6-ae13-4919-8e6e-4b06f29be1b3 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:18.622836 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 136/287] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:18 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:18.623711 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d793f6-ae13-4919-8e6e-4b06f29be1b3 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:18.679338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:18.710129 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334307b-2295-459e-abe9-5045761c1f6e None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:18.730764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d793f6-ae13-4919-8e6e-4b06f29be1b3 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:18.756880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2ba1d-7672-45c0-8540-78fde1543710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:18.757819 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2ba1d-7672-45c0-8540-78fde1543710 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:18.758108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2ba1d-7672-45c0-8540-78fde1543710 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:18.758385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2ba1d-7672-45c0-8540-78fde1543710 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:18.759422 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2ba1d-7672-45c0-8540-78fde1543710 None admin] RBAC: Adding query filter params (name=member) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:18.759759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2ba1d-7672-45c0-8540-78fde1543710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:18.761759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2ba1d-7672-45c0-8540-78fde1543710 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:18.767845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2272ed7-dc7e-4a50-aaf8-e88f81d98504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:18.768630 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2272ed7-dc7e-4a50-aaf8-e88f81d98504 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:18.769012 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2272ed7-dc7e-4a50-aaf8-e88f81d98504 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:18.769390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2272ed7-dc7e-4a50-aaf8-e88f81d98504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:18.771001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2272ed7-dc7e-4a50-aaf8-e88f81d98504 None admin] RBAC: Adding query filter params (role.id=a77e3ea4b46b4a3aaca8e814f53809b5, user.id=19a5838c8dfa415cbb6cd30897f8f54b, scope.system=all) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:18.771426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2272ed7-dc7e-4a50-aaf8-e88f81d98504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:18.772224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 152/288] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:18.772850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2ba1d-7672-45c0-8540-78fde1543710 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:18.773368 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2272ed7-dc7e-4a50-aaf8-e88f81d98504 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:18.780114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 137/289] 10.222.0.52 () {56 vars in 1308 bytes} [Thu May 19 13:40:18 2022] GET /identity/v3/role_assignments?user.id=19a5838c8dfa415cbb6cd30897f8f54b&scope.system=all&role.id=a77e3ea4b46b4a3aaca8e814f53809b5 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:18.781194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2272ed7-dc7e-4a50-aaf8-e88f81d98504 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:18.831718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:18.928274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2272ed7-dc7e-4a50-aaf8-e88f81d98504 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:18.966401 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5da0c0-14bb-4a1e-bf34-fcb6dd072892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:18.967339 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5da0c0-14bb-4a1e-bf34-fcb6dd072892 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:18.967628 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5da0c0-14bb-4a1e-bf34-fcb6dd072892 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:18.968012 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5da0c0-14bb-4a1e-bf34-fcb6dd072892 None admin] PATH_INFO: `/v3/users/598969ce89a04164ade8384e458f4c4b` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:18.969332 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5da0c0-14bb-4a1e-bf34-fcb6dd072892 None admin] RBAC: Authorizing `identity:get_user(user_id=598969ce89a04164ade8384e458f4c4b)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:18.970923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5da0c0-14bb-4a1e-bf34-fcb6dd072892 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:18.972605 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 138/290] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:18 2022] GET /identity/v3/users/598969ce89a04164ade8384e458f4c4b => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:18.980547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5da0c0-14bb-4a1e-bf34-fcb6dd072892 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:18.985986 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2ba1d-7672-45c0-8540-78fde1543710 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:19.018795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:19.093737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405064ca-3bd7-4af9-bac0-57b77ed78413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:19.095090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405064ca-3bd7-4af9-bac0-57b77ed78413 None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:19.095574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405064ca-3bd7-4af9-bac0-57b77ed78413 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:19.096075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405064ca-3bd7-4af9-bac0-57b77ed78413 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea/users/40d171854ae14801a9053fe7f688805d/roles/33252f8dd8084b9ea3fcaf16f4820401` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:19.097637 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:19.099480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405064ca-3bd7-4af9-bac0-57b77ed78413 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea, user_id=40d171854ae14801a9053fe7f688805d, role_id=33252f8dd8084b9ea3fcaf16f4820401)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:19.102001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405064ca-3bd7-4af9-bac0-57b77ed78413 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:19.112059 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5da0c0-14bb-4a1e-bf34-fcb6dd072892 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:19.118905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 153/291] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:40:18 2022] PUT /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea/users/40d171854ae14801a9053fe7f688805d/roles/33252f8dd8084b9ea3fcaf16f4820401 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:40:19.159576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b39d93-e8db-48d7-88f2-3b1ff71afe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:19.161276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b39d93-e8db-48d7-88f2-3b1ff71afe1f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:19.161800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b39d93-e8db-48d7-88f2-3b1ff71afe1f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:19.162284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b39d93-e8db-48d7-88f2-3b1ff71afe1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:19.257472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b39d93-e8db-48d7-88f2-3b1ff71afe1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:19.259441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b39d93-e8db-48d7-88f2-3b1ff71afe1f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:19.387590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:19.408733 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 139/292] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:40:19.414382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405064ca-3bd7-4af9-bac0-57b77ed78413 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:19.525314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405064ca-3bd7-4af9-bac0-57b77ed78413 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:19.565951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4f226-ad2a-4725-b584-223da39bd0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:19.566916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4f226-ad2a-4725-b584-223da39bd0a0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:19.567205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4f226-ad2a-4725-b584-223da39bd0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:19.567509 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4f226-ad2a-4725-b584-223da39bd0a0 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:19.571589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4f226-ad2a-4725-b584-223da39bd0a0 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:19.573048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4f226-ad2a-4725-b584-223da39bd0a0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:19.574769 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 154/293] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:19 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:19.580474 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b39d93-e8db-48d7-88f2-3b1ff71afe1f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:19.711595 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b39d93-e8db-48d7-88f2-3b1ff71afe1f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:19.764595 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88b688-939e-41a2-b7b2-5c3e6021eca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:19.765267 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88b688-939e-41a2-b7b2-5c3e6021eca8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:19.765555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88b688-939e-41a2-b7b2-5c3e6021eca8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:19.765834 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88b688-939e-41a2-b7b2-5c3e6021eca8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:19.766675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88b688-939e-41a2-b7b2-5c3e6021eca8 None admin] RBAC: Adding query filter params (role.id=a77e3ea4b46b4a3aaca8e814f53809b5, scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea, user.id=598969ce89a04164ade8384e458f4c4b) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:19.767093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88b688-939e-41a2-b7b2-5c3e6021eca8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:19.768541 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f88b688-939e-41a2-b7b2-5c3e6021eca8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:19.779611 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 140/294] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:19 2022] GET /identity/v3/role_assignments?user.id=598969ce89a04164ade8384e458f4c4b&scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea&role.id=a77e3ea4b46b4a3aaca8e814f53809b5 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:20.415374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19144f47-8ef9-448e-99bf-1b3037f349f2 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:20.415900 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19144f47-8ef9-448e-99bf-1b3037f349f2 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:20.415900 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19144f47-8ef9-448e-99bf-1b3037f349f2 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:20.417308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 155/295] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:20.423186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73a532-ca8b-481a-86a1-df52f59cd78e None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:20.423668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73a532-ca8b-481a-86a1-df52f59cd78e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:20.423743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73a532-ca8b-481a-86a1-df52f59cd78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:20.449382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-9d73a532-ca8b-481a-86a1-df52f59cd78e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:20.450586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9d73a532-ca8b-481a-86a1-df52f59cd78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:20.505166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:20.531185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 141/296] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:40:20.548002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19144f47-8ef9-448e-99bf-1b3037f349f2 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:20.582760 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81976bd1-878e-48d6-b423-a4764a454fd7 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:20.583470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81976bd1-878e-48d6-b423-a4764a454fd7 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:20.583955 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81976bd1-878e-48d6-b423-a4764a454fd7 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:20.585746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 142/297] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:20.591989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca3bc1-0418-4424-b37a-a17efb100ea5 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:20.592464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca3bc1-0418-4424-b37a-a17efb100ea5 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:20.592945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca3bc1-0418-4424-b37a-a17efb100ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:20.597608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:20.611259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-66ca3bc1-0418-4424-b37a-a17efb100ea5 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:20.612702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-66ca3bc1-0418-4424-b37a-a17efb100ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:20.670397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19144f47-8ef9-448e-99bf-1b3037f349f2 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:20.682736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 143/298] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:40:20.704244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca3bc1-0418-4424-b37a-a17efb100ea5 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:20.709006 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ae1d9-e80e-4fe6-8437-ad63e66594d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:20.710041 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ae1d9-e80e-4fe6-8437-ad63e66594d5 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:20.710524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ae1d9-e80e-4fe6-8437-ad63e66594d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:20.711029 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ae1d9-e80e-4fe6-8437-ad63e66594d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:20.722553 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ae1d9-e80e-4fe6-8437-ad63e66594d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-6f8ae1d9-e80e-4fe6-8437-ad63e66594d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:20.732179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:20.733134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:20.733924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:20.733924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:20.733924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:20.733924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:20.733924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:20.733924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:20.733924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:20.733924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:20.733924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:20.733924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:20.733924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:20.733924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:20.736890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 156/299] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:20.743767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ae1d9-e80e-4fe6-8437-ad63e66594d5 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:20.820675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca3bc1-0418-4424-b37a-a17efb100ea5 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:20.835959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:20.859232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f731a-af64-47e9-95a0-6c4d587d732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:20.860268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f731a-af64-47e9-95a0-6c4d587d732b None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:20.860697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f731a-af64-47e9-95a0-6c4d587d732b None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:20.861115 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f731a-af64-47e9-95a0-6c4d587d732b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:20.862455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f731a-af64-47e9-95a0-6c4d587d732b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:20.864305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f731a-af64-47e9-95a0-6c4d587d732b None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:20.870887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 144/300] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:20.872196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f731a-af64-47e9-95a0-6c4d587d732b None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:20.877165 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:20.881510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ae1d9-e80e-4fe6-8437-ad63e66594d5 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:20.922701 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7c60f-d996-4080-9484-f0374f7f3378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:20.923349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c60f-d996-4080-9484-f0374f7f3378 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:20.923633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c60f-d996-4080-9484-f0374f7f3378 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:20.923908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c60f-d996-4080-9484-f0374f7f3378 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:20.924658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7c60f-d996-4080-9484-f0374f7f3378 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:20.924965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7c60f-d996-4080-9484-f0374f7f3378 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:20.927546 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7c60f-d996-4080-9484-f0374f7f3378 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:20.943004 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 157/301] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:20.944592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7c60f-d996-4080-9484-f0374f7f3378 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:21.000422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f731a-af64-47e9-95a0-6c4d587d732b None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:21.037358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a94d6-07e3-4198-ae3e-ed89957f9977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:21.038026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a94d6-07e3-4198-ae3e-ed89957f9977 None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:21.038311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a94d6-07e3-4198-ae3e-ed89957f9977 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:21.038586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a94d6-07e3-4198-ae3e-ed89957f9977 None admin] PATH_INFO: `/v3/groups` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:21.039306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a94d6-07e3-4198-ae3e-ed89957f9977 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:21.041158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a94d6-07e3-4198-ae3e-ed89957f9977 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:21.057754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 145/302] 10.222.0.52 () {60 vars in 1134 bytes} [Thu May 19 13:40:20 2022] POST /identity/v3/groups => generated 229 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:40:21.080091 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7c60f-d996-4080-9484-f0374f7f3378 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:21.121513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81f41b-c800-4959-9224-ede996469999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:21.122810 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81f41b-c800-4959-9224-ede996469999 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:21.123324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81f41b-c800-4959-9224-ede996469999 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:21.123810 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81f41b-c800-4959-9224-ede996469999 None admin] PATH_INFO: `/v3/users/283e3d38f72749958f8e2c27e35ad884` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:21.139786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81f41b-c800-4959-9224-ede996469999 None admin] RBAC: Authorizing `identity:get_user(user_id=283e3d38f72749958f8e2c27e35ad884)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:21.141629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81f41b-c800-4959-9224-ede996469999 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:21.144046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 158/303] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:20 2022] GET /identity/v3/users/283e3d38f72749958f8e2c27e35ad884 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:21.150979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a94d6-07e3-4198-ae3e-ed89957f9977 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:21.200679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:21.270381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a94d6-07e3-4198-ae3e-ed89957f9977 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:21.310293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7afaf3-8160-4775-b4cc-b10b377421ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:21.311035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7afaf3-8160-4775-b4cc-b10b377421ce None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:21.311321 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7afaf3-8160-4775-b4cc-b10b377421ce None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:21.311597 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7afaf3-8160-4775-b4cc-b10b377421ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:21.312431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7afaf3-8160-4775-b4cc-b10b377421ce None admin] RBAC: Adding query filter params (role.id=33252f8dd8084b9ea3fcaf16f4820401, user.id=283e3d38f72749958f8e2c27e35ad884, scope.system=all) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:21.312743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7afaf3-8160-4775-b4cc-b10b377421ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:21.314191 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7afaf3-8160-4775-b4cc-b10b377421ce None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:21.321308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 146/304] 10.222.0.52 () {56 vars in 1308 bytes} [Thu May 19 13:40:21 2022] GET /identity/v3/role_assignments?user.id=283e3d38f72749958f8e2c27e35ad884&scope.system=all&role.id=33252f8dd8084b9ea3fcaf16f4820401 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:21.459146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bccc6-3a1b-47f4-b7ca-7ddbfd847356 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:21.459639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bccc6-3a1b-47f4-b7ca-7ddbfd847356 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:21.459639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bccc6-3a1b-47f4-b7ca-7ddbfd847356 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:21.461336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 159/305] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:21.467173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4019f8-16af-47a9-96fa-c27b05526d63 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:21.467693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4019f8-16af-47a9-96fa-c27b05526d63 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:21.467769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4019f8-16af-47a9-96fa-c27b05526d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:21.486376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-0e4019f8-16af-47a9-96fa-c27b05526d63 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:21.487626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-0e4019f8-16af-47a9-96fa-c27b05526d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:21.560902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 147/306] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:21.583932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bccc6-3a1b-47f4-b7ca-7ddbfd847356 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:21.608658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:21.703421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bccc6-3a1b-47f4-b7ca-7ddbfd847356 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:21.745747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbedd2d-5c24-42d1-a34b-6869f4f47542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:21.746720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbedd2d-5c24-42d1-a34b-6869f4f47542 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:21.747044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbedd2d-5c24-42d1-a34b-6869f4f47542 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:21.747316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbedd2d-5c24-42d1-a34b-6869f4f47542 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:21.759260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbedd2d-5c24-42d1-a34b-6869f4f47542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-5fbedd2d-5c24-42d1-a34b-6869f4f47542 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:21.767065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:21.767924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:21.768954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:21.768954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:21.768954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:21.768954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:21.768954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:21.768954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:21.768954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:21.768954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:21.768954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:21.768954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:21.768954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:21.768954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:21.768954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 160/307] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:21.776350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4019f8-16af-47a9-96fa-c27b05526d63 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:21.816852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:21.881860 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4019f8-16af-47a9-96fa-c27b05526d63 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:21.891497 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c0d3-cbdb-407a-a745-bb99d5c25c1d None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:21.891922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c0d3-cbdb-407a-a745-bb99d5c25c1d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:21.892305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c0d3-cbdb-407a-a745-bb99d5c25c1d None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:21.894242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 161/308] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:40:21.900364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff40b20-644d-4c1b-8e45-9941f6be87ad None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:21.900768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff40b20-644d-4c1b-8e45-9941f6be87ad None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:21.901149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff40b20-644d-4c1b-8e45-9941f6be87ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:21.919641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-cff40b20-644d-4c1b-8e45-9941f6be87ad None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:21.921257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-cff40b20-644d-4c1b-8e45-9941f6be87ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:21.921374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09fdf1-b7dd-447b-b556-24e177c0f1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:21.922004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09fdf1-b7dd-447b-b556-24e177c0f1c2 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:21.922283 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09fdf1-b7dd-447b-b556-24e177c0f1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:21.922554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09fdf1-b7dd-447b-b556-24e177c0f1c2 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:21.923323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a09fdf1-b7dd-447b-b556-24e177c0f1c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:21.923700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a09fdf1-b7dd-447b-b556-24e177c0f1c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:21.925284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a09fdf1-b7dd-447b-b556-24e177c0f1c2 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:21.934358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 148/309] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:21.940434 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09fdf1-b7dd-447b-b556-24e177c0f1c2 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:21.995188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 162/310] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:22.017942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff40b20-644d-4c1b-8e45-9941f6be87ad None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:22.031651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:22.058630 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:22.064253 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09fdf1-b7dd-447b-b556-24e177c0f1c2 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:22.099759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f82c3-1b75-423d-a639-d7ee7f8dc8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:22.100703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f82c3-1b75-423d-a639-d7ee7f8dc8bf None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:22.100917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f82c3-1b75-423d-a639-d7ee7f8dc8bf None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:22.101226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f82c3-1b75-423d-a639-d7ee7f8dc8bf None admin] PATH_INFO: `/v3/users/19a5838c8dfa415cbb6cd30897f8f54b` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:22.102510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f82c3-1b75-423d-a639-d7ee7f8dc8bf None admin] RBAC: Authorizing `identity:get_user(user_id=19a5838c8dfa415cbb6cd30897f8f54b)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:22.104513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f82c3-1b75-423d-a639-d7ee7f8dc8bf None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:22.110333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 149/311] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:21 2022] GET /identity/v3/users/19a5838c8dfa415cbb6cd30897f8f54b => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:22.112554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f82c3-1b75-423d-a639-d7ee7f8dc8bf None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:22.145878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff40b20-644d-4c1b-8e45-9941f6be87ad None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:22.184694 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0f5a6-a072-46be-acef-d2db647f31d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:22.185822 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0f5a6-a072-46be-acef-d2db647f31d1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:22.186403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0f5a6-a072-46be-acef-d2db647f31d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:22.186893 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0f5a6-a072-46be-acef-d2db647f31d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:22.199389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0f5a6-a072-46be-acef-d2db647f31d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-d7d0f5a6-a072-46be-acef-d2db647f31d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:22.208750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:22.209609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:22.210403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:22.210403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:22.210403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:22.210403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:22.210403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:22.210403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:22.210403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:22.210403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:22.210403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:22.210403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:22.210403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:22.210403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:22.213286 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 163/312] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:22.218480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0f5a6-a072-46be-acef-d2db647f31d1 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:22.229037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f82c3-1b75-423d-a639-d7ee7f8dc8bf None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:22.275566 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b9e59-cb18-49a7-b351-f1d5df26a0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:22.276429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b9e59-cb18-49a7-b351-f1d5df26a0c5 None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:22.276982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b9e59-cb18-49a7-b351-f1d5df26a0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:22.277271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b9e59-cb18-49a7-b351-f1d5df26a0c5 None admin] PATH_INFO: `/v3/system/users/19a5838c8dfa415cbb6cd30897f8f54b/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:22.279325 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b9e59-cb18-49a7-b351-f1d5df26a0c5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=19a5838c8dfa415cbb6cd30897f8f54b, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:22.281188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b9e59-cb18-49a7-b351-f1d5df26a0c5 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:22.298938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 150/313] 10.222.0.52 () {58 vars in 1334 bytes} [Thu May 19 13:40:22 2022] PUT /identity/v3/system/users/19a5838c8dfa415cbb6cd30897f8f54b/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:40:22.313724 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:22.353502 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0f5a6-a072-46be-acef-d2db647f31d1 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:22.396701 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262c293-f480-483a-b129-bb839323008d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:22.397737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262c293-f480-483a-b129-bb839323008d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:22.398217 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262c293-f480-483a-b129-bb839323008d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:22.398775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262c293-f480-483a-b129-bb839323008d None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:22.399787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262c293-f480-483a-b129-bb839323008d None admin] RBAC: Adding query filter params (name=reader) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:22.400527 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262c293-f480-483a-b129-bb839323008d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:22.406897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262c293-f480-483a-b129-bb839323008d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:22.412351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 164/314] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:22.418210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b9e59-cb18-49a7-b351-f1d5df26a0c5 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:22.451648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:22.533030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b9e59-cb18-49a7-b351-f1d5df26a0c5 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:22.571497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6185f73-efae-4638-afb1-b52e37f432aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:22.572363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6185f73-efae-4638-afb1-b52e37f432aa None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:22.572642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6185f73-efae-4638-afb1-b52e37f432aa None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:22.572910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6185f73-efae-4638-afb1-b52e37f432aa None admin] PATH_INFO: `/v3/users/40d171854ae14801a9053fe7f688805d` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:22.574093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6185f73-efae-4638-afb1-b52e37f432aa None admin] RBAC: Authorizing `identity:get_user(user_id=40d171854ae14801a9053fe7f688805d)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:22.575561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6185f73-efae-4638-afb1-b52e37f432aa None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:22.577496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 151/315] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:22 2022] GET /identity/v3/users/40d171854ae14801a9053fe7f688805d => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:22.583618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262c293-f480-483a-b129-bb839323008d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:22.685547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262c293-f480-483a-b129-bb839323008d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:22.714727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:22.719736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450623a-16bc-4611-9a90-24353891d37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:22.720309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450623a-16bc-4611-9a90-24353891d37d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:22.720594 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450623a-16bc-4611-9a90-24353891d37d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:22.720872 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450623a-16bc-4611-9a90-24353891d37d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:22.791877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450623a-16bc-4611-9a90-24353891d37d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:22.793246 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450623a-16bc-4611-9a90-24353891d37d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:22.893556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 165/316] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:22.899724 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6185f73-efae-4638-afb1-b52e37f432aa None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:23.001876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:23.009347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6185f73-efae-4638-afb1-b52e37f432aa None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:23.046245 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f3afb-cc1d-4880-8063-e05b0fc3175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:23.047059 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f3afb-cc1d-4880-8063-e05b0fc3175e None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:23.047341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f3afb-cc1d-4880-8063-e05b0fc3175e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:23.047618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f3afb-cc1d-4880-8063-e05b0fc3175e None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:23.048830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f3afb-cc1d-4880-8063-e05b0fc3175e None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:23.050380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f3afb-cc1d-4880-8063-e05b0fc3175e None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:23.052793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 152/317] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:22 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:23.058259 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450623a-16bc-4611-9a90-24353891d37d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:23.090447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:23.163941 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450623a-16bc-4611-9a90-24353891d37d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:23.200149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f136f-87e8-469a-933a-680c60d208ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:23.200800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f136f-87e8-469a-933a-680c60d208ed None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:23.201077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f136f-87e8-469a-933a-680c60d208ed None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:23.201353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f136f-87e8-469a-933a-680c60d208ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:23.202181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f136f-87e8-469a-933a-680c60d208ed None admin] RBAC: Adding query filter params (role.id=33252f8dd8084b9ea3fcaf16f4820401, scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea, user.id=40d171854ae14801a9053fe7f688805d) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:23.202490 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f136f-87e8-469a-933a-680c60d208ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:23.203955 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f136f-87e8-469a-933a-680c60d208ed None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:23.210485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 166/318] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:23 2022] GET /identity/v3/role_assignments?user.id=40d171854ae14801a9053fe7f688805d&scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea&role.id=33252f8dd8084b9ea3fcaf16f4820401 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:23.503207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a29be-e979-46a9-98cf-3b36f58e11db None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:23.503572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a29be-e979-46a9-98cf-3b36f58e11db None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:23.503765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a29be-e979-46a9-98cf-3b36f58e11db None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:23.507347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 153/319] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:23.513309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d0141-773e-45fb-97c5-d94b7dcca573 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:23.513628 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d0141-773e-45fb-97c5-d94b7dcca573 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:23.513888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d0141-773e-45fb-97c5-d94b7dcca573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:23.532237 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-5f3d0141-773e-45fb-97c5-d94b7dcca573 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:23.533432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5f3d0141-773e-45fb-97c5-d94b7dcca573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:23.604120 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 167/320] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:23.627001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a29be-e979-46a9-98cf-3b36f58e11db None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:23.695861 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38cebe-ebf1-4709-abf9-bd4759501010 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:23.696138 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38cebe-ebf1-4709-abf9-bd4759501010 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:23.696347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38cebe-ebf1-4709-abf9-bd4759501010 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:23.697719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 168/321] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:40:23.703227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa5204-c95c-4ca0-8eaa-2094a8613b36 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:23.703227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa5204-c95c-4ca0-8eaa-2094a8613b36 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:23.703455 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa5204-c95c-4ca0-8eaa-2094a8613b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:23.720156 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-4eaa5204-c95c-4ca0-8eaa-2094a8613b36 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:23.721339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4eaa5204-c95c-4ca0-8eaa-2094a8613b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:23.735091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a29be-e979-46a9-98cf-3b36f58e11db None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:23.742486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:23.775835 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8aeef-04b8-4a90-ae49-46b9bf3ec8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:23.776626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8aeef-04b8-4a90-ae49-46b9bf3ec8a8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:23.776903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8aeef-04b8-4a90-ae49-46b9bf3ec8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:23.777169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8aeef-04b8-4a90-ae49-46b9bf3ec8a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:23.778813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8aeef-04b8-4a90-ae49-46b9bf3ec8a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:23.780850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8aeef-04b8-4a90-ae49-46b9bf3ec8a8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:23.783368 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 154/322] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:23.788552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8aeef-04b8-4a90-ae49-46b9bf3ec8a8 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:23.792444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 169/323] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:23.813690 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa5204-c95c-4ca0-8eaa-2094a8613b36 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:23.821355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:23.895595 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8aeef-04b8-4a90-ae49-46b9bf3ec8a8 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:23.918216 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa5204-c95c-4ca0-8eaa-2094a8613b36 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:23.930298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f1433-4496-47b4-a170-985b6116e1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:23.931005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f1433-4496-47b4-a170-985b6116e1cd None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:23.931289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f1433-4496-47b4-a170-985b6116e1cd None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:23.931562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f1433-4496-47b4-a170-985b6116e1cd None admin] PATH_INFO: `/v3/groups` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:23.932251 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f1433-4496-47b4-a170-985b6116e1cd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:23.933681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f1433-4496-47b4-a170-985b6116e1cd None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:23.948640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 155/324] 10.222.0.52 () {60 vars in 1134 bytes} [Thu May 19 13:40:23 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:40:23.958247 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d34e4a-114e-4eac-ac4d-f7db97e857bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:23.959475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d34e4a-114e-4eac-ac4d-f7db97e857bb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:23.959766 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d34e4a-114e-4eac-ac4d-f7db97e857bb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:23.960048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d34e4a-114e-4eac-ac4d-f7db97e857bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:23.972571 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d34e4a-114e-4eac-ac4d-f7db97e857bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-b4d34e4a-114e-4eac-ac4d-f7db97e857bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:23.980995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:23.981967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:23.982798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:23.982798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:23.982798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:23.982798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:23.982798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:23.982798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:23.982798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:23.982798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:23.982798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:23.982798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:23.982798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:23.982798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:23.982798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 170/325] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:40:23.990428 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f1433-4496-47b4-a170-985b6116e1cd None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:24.098430 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:24.110295 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f1433-4496-47b4-a170-985b6116e1cd None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:24.149438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e772a-6832-4768-909e-9ac5ae2436e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:24.149642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e772a-6832-4768-909e-9ac5ae2436e4 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:24.149879 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e772a-6832-4768-909e-9ac5ae2436e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:24.150156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e772a-6832-4768-909e-9ac5ae2436e4 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:24.152194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e772a-6832-4768-909e-9ac5ae2436e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:24.152194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e772a-6832-4768-909e-9ac5ae2436e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:24.152606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e772a-6832-4768-909e-9ac5ae2436e4 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:24.162941 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 156/326] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:24.168425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d34e4a-114e-4eac-ac4d-f7db97e857bb None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:24.188311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:24.288178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d34e4a-114e-4eac-ac4d-f7db97e857bb None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:24.328798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54554616-e354-4c3a-92c5-d061967a4a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:24.330038 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54554616-e354-4c3a-92c5-d061967a4a79 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:24.330517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54554616-e354-4c3a-92c5-d061967a4a79 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:24.331017 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54554616-e354-4c3a-92c5-d061967a4a79 None admin] PATH_INFO: `/v3/users/283e3d38f72749958f8e2c27e35ad884` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:24.332661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54554616-e354-4c3a-92c5-d061967a4a79 None admin] RBAC: Authorizing `identity:get_user(user_id=283e3d38f72749958f8e2c27e35ad884)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:24.334410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54554616-e354-4c3a-92c5-d061967a4a79 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:24.336586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 171/327] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:24 2022] GET /identity/v3/users/283e3d38f72749958f8e2c27e35ad884 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:24.342356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e772a-6832-4768-909e-9ac5ae2436e4 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:24.463096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e772a-6832-4768-909e-9ac5ae2436e4 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:24.506898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe5d8e-2393-4a6a-8087-a431a6f5303a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:24.507826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe5d8e-2393-4a6a-8087-a431a6f5303a None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:24.508109 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe5d8e-2393-4a6a-8087-a431a6f5303a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:24.508381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe5d8e-2393-4a6a-8087-a431a6f5303a None admin] PATH_INFO: `/v3/system/users/283e3d38f72749958f8e2c27e35ad884/roles/33252f8dd8084b9ea3fcaf16f4820401` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:24.509989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe5d8e-2393-4a6a-8087-a431a6f5303a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=283e3d38f72749958f8e2c27e35ad884, role_id=33252f8dd8084b9ea3fcaf16f4820401)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:24.511424 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe5d8e-2393-4a6a-8087-a431a6f5303a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:24.522878 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 157/328] 10.222.0.52 () {58 vars in 1334 bytes} [Thu May 19 13:40:24 2022] PUT /identity/v3/system/users/283e3d38f72749958f8e2c27e35ad884/roles/33252f8dd8084b9ea3fcaf16f4820401 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:40:24.668236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468e19e-032c-400e-942d-0f4fa4584cbd None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:24.668791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468e19e-032c-400e-942d-0f4fa4584cbd None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:24.668791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468e19e-032c-400e-942d-0f4fa4584cbd None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:24.670576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 172/329] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:24.677647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10b169-67ed-4270-b6a7-61c96219a624 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:24.678143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10b169-67ed-4270-b6a7-61c96219a624 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:24.678217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10b169-67ed-4270-b6a7-61c96219a624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:24.700745 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-cf10b169-67ed-4270-b6a7-61c96219a624 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:24.702003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-cf10b169-67ed-4270-b6a7-61c96219a624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:24.720550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:24.785969 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 158/330] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:24.812868 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468e19e-032c-400e-942d-0f4fa4584cbd None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:24.912876 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:24.933050 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468e19e-032c-400e-942d-0f4fa4584cbd None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:24.975573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef89268-7620-4840-833a-3556506f4ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:24.976388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef89268-7620-4840-833a-3556506f4ded None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:24.976672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef89268-7620-4840-833a-3556506f4ded None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:24.976953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef89268-7620-4840-833a-3556506f4ded None admin] PATH_INFO: `/v3/roles/member` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:25.001176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef89268-7620-4840-833a-3556506f4ded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-2ef89268-7620-4840-833a-3556506f4ded None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:25.010395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:25.011288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:25.012106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:25.012106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:25.012106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:25.012106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:25.012106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:25.012106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:25.012106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:25.012106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:25.012106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:25.012106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:25.012106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:25.012106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:25.016418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 173/331] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:25.018892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10b169-67ed-4270-b6a7-61c96219a624 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:25.176421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10b169-67ed-4270-b6a7-61c96219a624 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:25.221796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f73e7d-78ea-4a58-b9bc-e00c14c8951d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:25.223442 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f73e7d-78ea-4a58-b9bc-e00c14c8951d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:25.224038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f73e7d-78ea-4a58-b9bc-e00c14c8951d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:25.224510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f73e7d-78ea-4a58-b9bc-e00c14c8951d None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:25.225471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f73e7d-78ea-4a58-b9bc-e00c14c8951d None admin] RBAC: Adding query filter params (name=member) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:25.226018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f73e7d-78ea-4a58-b9bc-e00c14c8951d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:25.227609 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f73e7d-78ea-4a58-b9bc-e00c14c8951d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:25.242458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 159/332] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:25.248544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef89268-7620-4840-833a-3556506f4ded None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:25.415315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef89268-7620-4840-833a-3556506f4ded None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:25.458416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943de70-d557-4671-a67f-9d87f1b3d045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:25.459317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943de70-d557-4671-a67f-9d87f1b3d045 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:25.459604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943de70-d557-4671-a67f-9d87f1b3d045 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:25.459880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943de70-d557-4671-a67f-9d87f1b3d045 None admin] PATH_INFO: `/v3/users/19a5838c8dfa415cbb6cd30897f8f54b` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:25.463608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943de70-d557-4671-a67f-9d87f1b3d045 None admin] RBAC: Authorizing `identity:get_user(user_id=19a5838c8dfa415cbb6cd30897f8f54b)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:25.465087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943de70-d557-4671-a67f-9d87f1b3d045 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:25.467124 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 174/333] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:25 2022] GET /identity/v3/users/19a5838c8dfa415cbb6cd30897f8f54b => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:25.474361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f73e7d-78ea-4a58-b9bc-e00c14c8951d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:25.497051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:25.625641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f73e7d-78ea-4a58-b9bc-e00c14c8951d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:25.676639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58d674-ca66-4a14-a8e5-8f87a1bed74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:25.677486 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58d674-ca66-4a14-a8e5-8f87a1bed74a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:25.677766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58d674-ca66-4a14-a8e5-8f87a1bed74a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:25.678390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58d674-ca66-4a14-a8e5-8f87a1bed74a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:25.678874 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58d674-ca66-4a14-a8e5-8f87a1bed74a None admin] RBAC: Adding query filter params (role.id=a77e3ea4b46b4a3aaca8e814f53809b5, user.id=19a5838c8dfa415cbb6cd30897f8f54b, scope.system=all) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:25.679193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58d674-ca66-4a14-a8e5-8f87a1bed74a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:25.680611 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58d674-ca66-4a14-a8e5-8f87a1bed74a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:25.698976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 160/334] 10.222.0.52 () {56 vars in 1308 bytes} [Thu May 19 13:40:25 2022] GET /identity/v3/role_assignments?user.id=19a5838c8dfa415cbb6cd30897f8f54b&scope.system=all&role.id=a77e3ea4b46b4a3aaca8e814f53809b5 => generated 510 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:26.818440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007fb0a-6d3c-4441-a78f-039648d02ff2 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:26.818942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007fb0a-6d3c-4441-a78f-039648d02ff2 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:26.819020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007fb0a-6d3c-4441-a78f-039648d02ff2 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:26.820774 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 175/335] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:26.825952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718b875-108f-47a0-8c33-0c3e791c26fb None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:26.826814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718b875-108f-47a0-8c33-0c3e791c26fb None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:26.827326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718b875-108f-47a0-8c33-0c3e791c26fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:26.846399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-0718b875-108f-47a0-8c33-0c3e791c26fb None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:26.848791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-0718b875-108f-47a0-8c33-0c3e791c26fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:26.921619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:26.929491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 161/336] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:26.966387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007fb0a-6d3c-4441-a78f-039648d02ff2 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:26.983470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:27.095637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007fb0a-6d3c-4441-a78f-039648d02ff2 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:27.137903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d641a89-b8c5-4bce-bfbc-51dae55b81ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:27.139084 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d641a89-b8c5-4bce-bfbc-51dae55b81ec None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:27.139617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d641a89-b8c5-4bce-bfbc-51dae55b81ec None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:27.140096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d641a89-b8c5-4bce-bfbc-51dae55b81ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:27.158811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d641a89-b8c5-4bce-bfbc-51dae55b81ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-7d641a89-b8c5-4bce-bfbc-51dae55b81ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:27.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:27.169103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:27.169877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:27.169877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:27.169877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:27.169877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:27.169877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:27.169877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:27.169877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:27.169877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:27.169877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:27.169877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:27.169877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:27.169877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:27.178177 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718b875-108f-47a0-8c33-0c3e791c26fb None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:27.180516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 176/337] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:27.295067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718b875-108f-47a0-8c33-0c3e791c26fb None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:27.346923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558531c-a164-4646-a1b1-feb363107ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:27.347968 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558531c-a164-4646-a1b1-feb363107ff5 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:27.349164 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558531c-a164-4646-a1b1-feb363107ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:27.349651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558531c-a164-4646-a1b1-feb363107ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:27.350618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558531c-a164-4646-a1b1-feb363107ff5 None admin] RBAC: Adding query filter params (name=member) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:27.351150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558531c-a164-4646-a1b1-feb363107ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:27.352723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558531c-a164-4646-a1b1-feb363107ff5 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:27.367913 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 162/338] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:27.373285 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d641a89-b8c5-4bce-bfbc-51dae55b81ec None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:27.492681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:27.532479 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d641a89-b8c5-4bce-bfbc-51dae55b81ec None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:27.581753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12fb83-5022-4b70-88c8-e95d3f9755d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:27.583718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12fb83-5022-4b70-88c8-e95d3f9755d2 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:27.584016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12fb83-5022-4b70-88c8-e95d3f9755d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:27.584301 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12fb83-5022-4b70-88c8-e95d3f9755d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:27.668136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12fb83-5022-4b70-88c8-e95d3f9755d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:27.669594 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12fb83-5022-4b70-88c8-e95d3f9755d2 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:27.736661 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726acaf-1c31-4150-9334-0e75ff37e09f None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:27.736989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726acaf-1c31-4150-9334-0e75ff37e09f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:27.737233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726acaf-1c31-4150-9334-0e75ff37e09f None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:27.739797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 163/339] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:27.745173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44cae1-4004-4666-bd12-9606cc69df22 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:27.745458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44cae1-4004-4666-bd12-9606cc69df22 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:27.745735 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44cae1-4004-4666-bd12-9606cc69df22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:27.765677 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:27.787367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-2f44cae1-4004-4666-bd12-9606cc69df22 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:27.788590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2f44cae1-4004-4666-bd12-9606cc69df22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:27.839280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 177/340] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:27.844961 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12fb83-5022-4b70-88c8-e95d3f9755d2 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:27.921898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:27.943404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 164/341] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:27.944979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f3af6-2950-4e12-9c11-e4de45640633 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:27.945410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f3af6-2950-4e12-9c11-e4de45640633 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:27.945857 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f3af6-2950-4e12-9c11-e4de45640633 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:27.947554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 165/342] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:27.953247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff2f46-bebc-4cd3-aa27-ea51367fecc6 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:27.953787 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff2f46-bebc-4cd3-aa27-ea51367fecc6 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:27.954260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff2f46-bebc-4cd3-aa27-ea51367fecc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:27.987915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-1aff2f46-bebc-4cd3-aa27-ea51367fecc6 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:27.990124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1aff2f46-bebc-4cd3-aa27-ea51367fecc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:28.003115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12fb83-5022-4b70-88c8-e95d3f9755d2 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:28.053841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d721a-a1ef-4622-8338-16bb98e3715b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:28.056408 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d721a-a1ef-4622-8338-16bb98e3715b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:28.056914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d721a-a1ef-4622-8338-16bb98e3715b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:28.057371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d721a-a1ef-4622-8338-16bb98e3715b None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:28.058792 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d721a-a1ef-4622-8338-16bb98e3715b None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:28.060838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d721a-a1ef-4622-8338-16bb98e3715b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:28.065721 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 178/343] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:27 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:28.067585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d721a-a1ef-4622-8338-16bb98e3715b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:28.150458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 166/344] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:40:28.151934 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff2f46-bebc-4cd3-aa27-ea51367fecc6 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:28.158805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:28.256644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d721a-a1ef-4622-8338-16bb98e3715b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:28.300692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39fcc7-ccd7-4704-bf4c-ed686398a678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:28.302814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39fcc7-ccd7-4704-bf4c-ed686398a678 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:28.303133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39fcc7-ccd7-4704-bf4c-ed686398a678 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:28.303444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39fcc7-ccd7-4704-bf4c-ed686398a678 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:28.318919 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39fcc7-ccd7-4704-bf4c-ed686398a678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-bf39fcc7-ccd7-4704-bf4c-ed686398a678 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:28.326514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:28.327485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:28.328325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:28.328325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:28.328325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:28.328325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:28.328325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:28.328325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:28.328325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:28.328325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:28.328325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:28.328325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:28.328325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:40:28.328325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:28.328325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 179/345] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:28.333772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39fcc7-ccd7-4704-bf4c-ed686398a678 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:28.359277 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff2f46-bebc-4cd3-aa27-ea51367fecc6 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:28.386359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:28.414178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5a2ca-3742-4b73-85a8-790432242689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:28.415273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a2ca-3742-4b73-85a8-790432242689 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:28.415894 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a2ca-3742-4b73-85a8-790432242689 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:28.416198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a2ca-3742-4b73-85a8-790432242689 None admin] PATH_INFO: `/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:28.427606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5a2ca-3742-4b73-85a8-790432242689 None admin] RBAC: Authorizing `identity:get_group(group_id=4bdb1f914c0b4f7f99e863084f9e1806)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:28.429671 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5a2ca-3742-4b73-85a8-790432242689 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:28.433140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 167/346] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:28 2022] GET /identity/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:28.434447 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd19fd-7db9-4788-9936-35d9bead7f13 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:28.434739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd19fd-7db9-4788-9936-35d9bead7f13 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:28.435064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd19fd-7db9-4788-9936-35d9bead7f13 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:28.436667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 168/347] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:28.437393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd19fd-7db9-4788-9936-35d9bead7f13 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:28.480070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:28.484473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39fcc7-ccd7-4704-bf4c-ed686398a678 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:28.540145 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a39e32-6b4f-4a22-85aa-ec70012b7925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:28.541342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a39e32-6b4f-4a22-85aa-ec70012b7925 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:28.541743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a39e32-6b4f-4a22-85aa-ec70012b7925 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:28.542027 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a39e32-6b4f-4a22-85aa-ec70012b7925 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:28.555622 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a39e32-6b4f-4a22-85aa-ec70012b7925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-b3a39e32-6b4f-4a22-85aa-ec70012b7925 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:28.567299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:28.568430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:28.569238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:28.569238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:28.569238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:28.569238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:28.569238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:28.569238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:28.569238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:28.569238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:28.569238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:28.569238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:28.569238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:28.569238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:28.569238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 180/348] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:40:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:28.573093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a39e32-6b4f-4a22-85aa-ec70012b7925 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:28.606189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd19fd-7db9-4788-9936-35d9bead7f13 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:28.653456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5bed2-cb25-4e4e-9883-93c6677aba06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:28.654188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5bed2-cb25-4e4e-9883-93c6677aba06 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:28.654473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5bed2-cb25-4e4e-9883-93c6677aba06 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:28.654751 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5bed2-cb25-4e4e-9883-93c6677aba06 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:28.655884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5bed2-cb25-4e4e-9883-93c6677aba06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:28.656194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5bed2-cb25-4e4e-9883-93c6677aba06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:28.657569 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5bed2-cb25-4e4e-9883-93c6677aba06 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:28.668846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 169/349] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:28.670290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016434c-cf88-4c04-b954-125c2404f19b None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:28.670623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016434c-cf88-4c04-b954-125c2404f19b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:28.674113 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016434c-cf88-4c04-b954-125c2404f19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:28.694277 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-0016434c-cf88-4c04-b954-125c2404f19b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:28.695507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-0016434c-cf88-4c04-b954-125c2404f19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:28.709695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a39e32-6b4f-4a22-85aa-ec70012b7925 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:28.753935 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:28.763055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66e43e-e7d1-45a5-a049-768537fb69c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:28.763729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e43e-e7d1-45a5-a049-768537fb69c9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:28.764017 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e43e-e7d1-45a5-a049-768537fb69c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:28.764300 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e43e-e7d1-45a5-a049-768537fb69c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:28.765213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66e43e-e7d1-45a5-a049-768537fb69c9 None admin] RBAC: Adding query filter params (group.id=4bdb1f914c0b4f7f99e863084f9e1806, role.id=a77e3ea4b46b4a3aaca8e814f53809b5, scope.project.id=d9fbb046f448445799c63c2b7d78bd93) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:28.765530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66e43e-e7d1-45a5-a049-768537fb69c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:28.767024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66e43e-e7d1-45a5-a049-768537fb69c9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:28.777025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 181/350] 10.222.0.52 () {56 vars in 1376 bytes} [Thu May 19 13:40:28 2022] GET /identity/v3/role_assignments?group.id=4bdb1f914c0b4f7f99e863084f9e1806&scope.project.id=d9fbb046f448445799c63c2b7d78bd93&role.id=a77e3ea4b46b4a3aaca8e814f53809b5 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:28.777968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66e43e-e7d1-45a5-a049-768537fb69c9 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:28.788709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 170/351] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:28.796500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016434c-cf88-4c04-b954-125c2404f19b None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:28.863715 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:29.017548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66e43e-e7d1-45a5-a049-768537fb69c9 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:29.022091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016434c-cf88-4c04-b954-125c2404f19b None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:29.065153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4801f37-59b0-4f49-a68b-ab935d77b9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:29.066057 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4801f37-59b0-4f49-a68b-ab935d77b9d3 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:29.066342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4801f37-59b0-4f49-a68b-ab935d77b9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:29.067375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4801f37-59b0-4f49-a68b-ab935d77b9d3 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:29.068255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4801f37-59b0-4f49-a68b-ab935d77b9d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:29.068580 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4801f37-59b0-4f49-a68b-ab935d77b9d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:29.070008 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4801f37-59b0-4f49-a68b-ab935d77b9d3 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:29.071792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c02cb9-4c0f-4f23-8304-107421741324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:29.072706 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c02cb9-4c0f-4f23-8304-107421741324 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:29.072988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c02cb9-4c0f-4f23-8304-107421741324 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:29.073261 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c02cb9-4c0f-4f23-8304-107421741324 None admin] PATH_INFO: `/v3/users/283e3d38f72749958f8e2c27e35ad884` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:29.075527 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c02cb9-4c0f-4f23-8304-107421741324 None admin] RBAC: Authorizing `identity:get_user(user_id=283e3d38f72749958f8e2c27e35ad884)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:29.077011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c02cb9-4c0f-4f23-8304-107421741324 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:29.078821 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 171/352] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:40:28 2022] GET /identity/v3/users/283e3d38f72749958f8e2c27e35ad884 => generated 293 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:29.080352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c02cb9-4c0f-4f23-8304-107421741324 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:29.083111 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 182/353] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:29.086251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4801f37-59b0-4f49-a68b-ab935d77b9d3 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:29.163780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:29.231338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:29.244106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c02cb9-4c0f-4f23-8304-107421741324 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:29.244290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4801f37-59b0-4f49-a68b-ab935d77b9d3 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:29.283361 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22d1bb-b366-4c22-b7fd-21eef06103c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:29.284032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22d1bb-b366-4c22-b7fd-21eef06103c2 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:29.284325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22d1bb-b366-4c22-b7fd-21eef06103c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:29.284612 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22d1bb-b366-4c22-b7fd-21eef06103c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:29.286464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22d1bb-b366-4c22-b7fd-21eef06103c2 None admin] RBAC: Adding query filter params (role.id=33252f8dd8084b9ea3fcaf16f4820401, user.id=283e3d38f72749958f8e2c27e35ad884, scope.system=all) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:29.286904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22d1bb-b366-4c22-b7fd-21eef06103c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:29.288707 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22d1bb-b366-4c22-b7fd-21eef06103c2 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:29.292208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60630e50-90f6-444d-8275-19f0c84ee1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:29.293280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60630e50-90f6-444d-8275-19f0c84ee1dd None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:29.293570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60630e50-90f6-444d-8275-19f0c84ee1dd None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:29.293848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60630e50-90f6-444d-8275-19f0c84ee1dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:29.304193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 183/354] 10.222.0.52 () {56 vars in 1308 bytes} [Thu May 19 13:40:29 2022] GET /identity/v3/role_assignments?user.id=283e3d38f72749958f8e2c27e35ad884&scope.system=all&role.id=33252f8dd8084b9ea3fcaf16f4820401 => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:29.305614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e689f-1bce-4c54-bb63-e9260e28451c None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:29.305913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e689f-1bce-4c54-bb63-e9260e28451c None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:29.306198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e689f-1bce-4c54-bb63-e9260e28451c None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:29.307879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 184/355] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:29.308670 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e689f-1bce-4c54-bb63-e9260e28451c None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:29.316829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60630e50-90f6-444d-8275-19f0c84ee1dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-60630e50-90f6-444d-8275-19f0c84ee1dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:29.324898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:29.325777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:29.326551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:29.326551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:29.326551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:29.326551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:29.326551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:29.326551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:29.326551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:29.326551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:29.326551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:29.326551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:29.326551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:29.326551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:29.326551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 172/356] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:29.327743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5257ce-883e-4ff1-8624-5782de9a6a7e None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:29.327743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5257ce-883e-4ff1-8624-5782de9a6a7e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:29.328032 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5257ce-883e-4ff1-8624-5782de9a6a7e None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:29.329549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 173/357] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:29.330696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031cd09-6364-4029-b122-9def960d1c28 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:29.330990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031cd09-6364-4029-b122-9def960d1c28 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:29.331275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031cd09-6364-4029-b122-9def960d1c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:29.352147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-d031cd09-6364-4029-b122-9def960d1c28 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:29.353402 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d031cd09-6364-4029-b122-9def960d1c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:29.444570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 174/358] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:29.446136 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0df100-0ab5-483a-9b35-f0b13d47679f None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:29.446432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0df100-0ab5-483a-9b35-f0b13d47679f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:29.446698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0df100-0ab5-483a-9b35-f0b13d47679f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:29.458703 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e689f-1bce-4c54-bb63-e9260e28451c None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:29.466811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-0c0df100-0ab5-483a-9b35-f0b13d47679f None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:29.468283 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-0c0df100-0ab5-483a-9b35-f0b13d47679f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:29.501662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:29.504697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a6594-bf27-4193-b760-1240f48651aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:29.505653 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a6594-bf27-4193-b760-1240f48651aa None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:29.506266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a6594-bf27-4193-b760-1240f48651aa None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:29.506729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a6594-bf27-4193-b760-1240f48651aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:29.527194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:29.555196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 175/359] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:29.556328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0df100-0ab5-483a-9b35-f0b13d47679f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:29.614108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a6594-bf27-4193-b760-1240f48651aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:29.615662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a6594-bf27-4193-b760-1240f48651aa None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:29.690000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0df100-0ab5-483a-9b35-f0b13d47679f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:29.734689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 185/360] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:29.735585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a6594-bf27-4193-b760-1240f48651aa None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:29.742166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30465a-e026-4208-9008-c0692808b974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:29.742953 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30465a-e026-4208-9008-c0692808b974 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:29.743179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30465a-e026-4208-9008-c0692808b974 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:29.743461 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30465a-e026-4208-9008-c0692808b974 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:29.744232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30465a-e026-4208-9008-c0692808b974 None admin] RBAC: Adding query filter params (name=member) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:29.744544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30465a-e026-4208-9008-c0692808b974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:29.745935 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30465a-e026-4208-9008-c0692808b974 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:29.757274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 176/361] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:29.758167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30465a-e026-4208-9008-c0692808b974 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:29.795155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:29.802180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:29.870482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a6594-bf27-4193-b760-1240f48651aa None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:29.903414 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30465a-e026-4208-9008-c0692808b974 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:29.912838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656342cf-f536-45be-b5a7-6036f98e6757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:29.913728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656342cf-f536-45be-b5a7-6036f98e6757 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:29.914073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656342cf-f536-45be-b5a7-6036f98e6757 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:29.914579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656342cf-f536-45be-b5a7-6036f98e6757 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:29.929490 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656342cf-f536-45be-b5a7-6036f98e6757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-656342cf-f536-45be-b5a7-6036f98e6757 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:29.940903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:29.942290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:29.943315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:29.943315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:29.943315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:29.943315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:29.943315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:29.943315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:29.943315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:29.943315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:29.943315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:29.943315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:29.943315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:29.943315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:29.945678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 186/362] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:40:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:29.947031 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656342cf-f536-45be-b5a7-6036f98e6757 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:29.952490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05940feb-fe26-4c3a-bb37-f26c728af8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:29.953449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05940feb-fe26-4c3a-bb37-f26c728af8c6 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:29.953880 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05940feb-fe26-4c3a-bb37-f26c728af8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:29.954158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05940feb-fe26-4c3a-bb37-f26c728af8c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:29.966535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05940feb-fe26-4c3a-bb37-f26c728af8c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-05940feb-fe26-4c3a-bb37-f26c728af8c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:29.975930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:29.976841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:29.977618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:29.977618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:29.977618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:29.977618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:29.977618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:29.977618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:29.977618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:29.977618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:29.977618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:29.977618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:29.977618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:29.977618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:29.977618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 177/363] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:40:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:29.978275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05940feb-fe26-4c3a-bb37-f26c728af8c6 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:30.086880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656342cf-f536-45be-b5a7-6036f98e6757 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:30.087071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:30.089334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:30.113742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05940feb-fe26-4c3a-bb37-f26c728af8c6 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:30.147478 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d517db2-df0f-4a55-b8f3-508bb54dc6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:30.148642 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d517db2-df0f-4a55-b8f3-508bb54dc6d9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:30.149085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d517db2-df0f-4a55-b8f3-508bb54dc6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:30.149528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d517db2-df0f-4a55-b8f3-508bb54dc6d9 None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:30.152369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d517db2-df0f-4a55-b8f3-508bb54dc6d9 None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:30.153860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d517db2-df0f-4a55-b8f3-508bb54dc6d9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:30.154431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80916f-f493-443f-9eea-90d00f947cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:30.159889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80916f-f493-443f-9eea-90d00f947cf0 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:30.159889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80916f-f493-443f-9eea-90d00f947cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:30.160781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 187/364] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:29 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:30.160922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80916f-f493-443f-9eea-90d00f947cf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:30.161047 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d517db2-df0f-4a55-b8f3-508bb54dc6d9 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:30.248245 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80916f-f493-443f-9eea-90d00f947cf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:30.248979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80916f-f493-443f-9eea-90d00f947cf0 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:30.279835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d517db2-df0f-4a55-b8f3-508bb54dc6d9 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:30.319741 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f11401-a7ae-4077-b784-3e24e9381e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:30.320418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f11401-a7ae-4077-b784-3e24e9381e8b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:30.320702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f11401-a7ae-4077-b784-3e24e9381e8b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:30.320979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f11401-a7ae-4077-b784-3e24e9381e8b None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:30.321737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f11401-a7ae-4077-b784-3e24e9381e8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:30.322046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f11401-a7ae-4077-b784-3e24e9381e8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:30.323932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f11401-a7ae-4077-b784-3e24e9381e8b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:30.334553 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 188/365] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:30.335562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f11401-a7ae-4077-b784-3e24e9381e8b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:30.356923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:30.374441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 178/366] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:30.380586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80916f-f493-443f-9eea-90d00f947cf0 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:30.380744 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:30.484464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f11401-a7ae-4077-b784-3e24e9381e8b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:30.514104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80916f-f493-443f-9eea-90d00f947cf0 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:30.538544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76605e91-8331-4000-9d21-655ecede0936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:30.539278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76605e91-8331-4000-9d21-655ecede0936 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:30.539564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76605e91-8331-4000-9d21-655ecede0936 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:30.539843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76605e91-8331-4000-9d21-655ecede0936 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:30.541055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76605e91-8331-4000-9d21-655ecede0936 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:30.541527 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76605e91-8331-4000-9d21-655ecede0936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:30.543415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76605e91-8331-4000-9d21-655ecede0936 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:30.554564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 189/367] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:30.555451 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76605e91-8331-4000-9d21-655ecede0936 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:30.564762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ce9b8-c446-470e-b6f3-0da18c8befc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:30.565625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ce9b8-c446-470e-b6f3-0da18c8befc4 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:30.565908 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ce9b8-c446-470e-b6f3-0da18c8befc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:30.566181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ce9b8-c446-470e-b6f3-0da18c8befc4 None admin] PATH_INFO: `/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:30.568071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ce9b8-c446-470e-b6f3-0da18c8befc4 None admin] RBAC: Authorizing `identity:get_group(group_id=4bdb1f914c0b4f7f99e863084f9e1806)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:30.571082 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ce9b8-c446-470e-b6f3-0da18c8befc4 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:30.573157 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 179/368] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:30 2022] GET /identity/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:30.574380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ce9b8-c446-470e-b6f3-0da18c8befc4 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:30.647156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:30.694294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:30.701089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ce9b8-c446-470e-b6f3-0da18c8befc4 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:30.701493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76605e91-8331-4000-9d21-655ecede0936 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:30.748221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb1e47-45e2-4008-b1c4-2010860857a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:30.748457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376b081-cefc-4fdb-9bc7-c325051e51f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:30.748457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376b081-cefc-4fdb-9bc7-c325051e51f2 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:30.748607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb1e47-45e2-4008-b1c4-2010860857a8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:30.748676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376b081-cefc-4fdb-9bc7-c325051e51f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:30.748752 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb1e47-45e2-4008-b1c4-2010860857a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:30.748821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376b081-cefc-4fdb-9bc7-c325051e51f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:30.748894 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb1e47-45e2-4008-b1c4-2010860857a8 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:30.750473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb1e47-45e2-4008-b1c4-2010860857a8 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:30.751684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb1e47-45e2-4008-b1c4-2010860857a8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:30.753394 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 180/369] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:30 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:30.755712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb1e47-45e2-4008-b1c4-2010860857a8 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:30.846043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376b081-cefc-4fdb-9bc7-c325051e51f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:30.847556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376b081-cefc-4fdb-9bc7-c325051e51f2 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:30.889814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb1e47-45e2-4008-b1c4-2010860857a8 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:30.916381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:30.934210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc1b1d-f0f7-4077-b29f-624e017d9b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:30.934910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc1b1d-f0f7-4077-b29f-624e017d9b07 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:30.935201 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc1b1d-f0f7-4077-b29f-624e017d9b07 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:30.935475 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc1b1d-f0f7-4077-b29f-624e017d9b07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:30.971396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:30.981914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 190/370] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:30.983956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376b081-cefc-4fdb-9bc7-c325051e51f2 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:31.018037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc1b1d-f0f7-4077-b29f-624e017d9b07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:31.020929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc1b1d-f0f7-4077-b29f-624e017d9b07 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:31.130430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376b081-cefc-4fdb-9bc7-c325051e51f2 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:31.137374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 181/371] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:31.138819 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc1b1d-f0f7-4077-b29f-624e017d9b07 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:31.167434 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:31.176305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e682c1-1571-47ad-aedb-bf2cb6f9ad11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:31.177383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e682c1-1571-47ad-aedb-bf2cb6f9ad11 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:31.177726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e682c1-1571-47ad-aedb-bf2cb6f9ad11 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:31.178009 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e682c1-1571-47ad-aedb-bf2cb6f9ad11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:31.178839 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e682c1-1571-47ad-aedb-bf2cb6f9ad11 None admin] RBAC: Adding query filter params (group.id=4bdb1f914c0b4f7f99e863084f9e1806, role.id=5c2125e2125d40bca6590d50afd677d1, scope.project.id=d9fbb046f448445799c63c2b7d78bd93) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:31.179214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e682c1-1571-47ad-aedb-bf2cb6f9ad11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:31.181002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e682c1-1571-47ad-aedb-bf2cb6f9ad11 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:31.189047 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 191/372] 10.222.0.52 () {56 vars in 1376 bytes} [Thu May 19 13:40:30 2022] GET /identity/v3/role_assignments?group.id=4bdb1f914c0b4f7f99e863084f9e1806&scope.project.id=d9fbb046f448445799c63c2b7d78bd93&role.id=5c2125e2125d40bca6590d50afd677d1 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:31.192261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e682c1-1571-47ad-aedb-bf2cb6f9ad11 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:31.277710 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc1b1d-f0f7-4077-b29f-624e017d9b07 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:31.303213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:31.327495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ae182-e4f6-4eca-8bfb-8a5233046c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:31.331145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ae182-e4f6-4eca-8bfb-8a5233046c27 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:31.331145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ae182-e4f6-4eca-8bfb-8a5233046c27 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:31.331145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ae182-e4f6-4eca-8bfb-8a5233046c27 None admin] PATH_INFO: `/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:31.331145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ae182-e4f6-4eca-8bfb-8a5233046c27 None admin] RBAC: Authorizing `identity:get_group(group_id=4bdb1f914c0b4f7f99e863084f9e1806)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:31.333605 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ae182-e4f6-4eca-8bfb-8a5233046c27 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:31.335775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 182/373] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:31 2022] GET /identity/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:31.336583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ae182-e4f6-4eca-8bfb-8a5233046c27 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:31.361542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e682c1-1571-47ad-aedb-bf2cb6f9ad11 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:31.406649 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf373c2-9b1f-40b2-bc16-db8aa4f0b0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:31.408153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf373c2-9b1f-40b2-bc16-db8aa4f0b0c9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:31.408153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf373c2-9b1f-40b2-bc16-db8aa4f0b0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:31.408347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf373c2-9b1f-40b2-bc16-db8aa4f0b0c9 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:31.410850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf373c2-9b1f-40b2-bc16-db8aa4f0b0c9 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:31.412346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf373c2-9b1f-40b2-bc16-db8aa4f0b0c9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:31.414135 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 192/374] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:31 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:31.415358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8854e-9865-4592-88ef-9a9e3a296caf None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:31.415643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8854e-9865-4592-88ef-9a9e3a296caf None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:31.415920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8854e-9865-4592-88ef-9a9e3a296caf None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:31.417403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 193/375] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:31.418213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8854e-9865-4592-88ef-9a9e3a296caf None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:31.466196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:31.482506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ae182-e4f6-4eca-8bfb-8a5233046c27 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:31.532001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ad767-4145-4bf2-9d22-63d584aea169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:31.533480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ad767-4145-4bf2-9d22-63d584aea169 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:31.533930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ad767-4145-4bf2-9d22-63d584aea169 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:31.534196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ad767-4145-4bf2-9d22-63d584aea169 None admin] PATH_INFO: `/v3/projects/2f0d02746fbd4bebabb402313125d4b8` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:31.536053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ad767-4145-4bf2-9d22-63d584aea169 None admin] RBAC: Authorizing `identity:get_project(project_id=2f0d02746fbd4bebabb402313125d4b8)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:31.538266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ad767-4145-4bf2-9d22-63d584aea169 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:31.540168 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 183/376] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:31 2022] GET /identity/v3/projects/2f0d02746fbd4bebabb402313125d4b8 => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:31.541870 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ea71b-a150-44a0-8ae4-6c5fd39be7c1 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:31.542152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ea71b-a150-44a0-8ae4-6c5fd39be7c1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:31.542426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ea71b-a150-44a0-8ae4-6c5fd39be7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:31.564832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-b85ea71b-a150-44a0-8ae4-6c5fd39be7c1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:31.566087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b85ea71b-a150-44a0-8ae4-6c5fd39be7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:31.568703 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8854e-9865-4592-88ef-9a9e3a296caf None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:31.604897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:31.615852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c118b-fbc1-4f58-bf62-6c17f12b2aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:31.616790 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c118b-fbc1-4f58-bf62-6c17f12b2aee None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:31.617263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c118b-fbc1-4f58-bf62-6c17f12b2aee None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:31.617548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c118b-fbc1-4f58-bf62-6c17f12b2aee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:31.618384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c118b-fbc1-4f58-bf62-6c17f12b2aee None admin] RBAC: Adding query filter params (group.id=4bdb1f914c0b4f7f99e863084f9e1806, role.id=a77e3ea4b46b4a3aaca8e814f53809b5, scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:31.618698 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c118b-fbc1-4f58-bf62-6c17f12b2aee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:31.620200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c118b-fbc1-4f58-bf62-6c17f12b2aee None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:31.630912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 194/377] 10.222.0.52 () {56 vars in 1376 bytes} [Thu May 19 13:40:31 2022] GET /identity/v3/role_assignments?group.id=4bdb1f914c0b4f7f99e863084f9e1806&scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea&role.id=a77e3ea4b46b4a3aaca8e814f53809b5 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:31.630912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c118b-fbc1-4f58-bf62-6c17f12b2aee None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:31.664327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 184/378] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:31.665485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ea71b-a150-44a0-8ae4-6c5fd39be7c1 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:31.791432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c118b-fbc1-4f58-bf62-6c17f12b2aee None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:31.801441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:31.820586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ea71b-a150-44a0-8ae4-6c5fd39be7c1 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:31.837192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa9cb2-6a31-484d-8631-a3b158b012a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:31.838067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9cb2-6a31-484d-8631-a3b158b012a2 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:31.838354 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9cb2-6a31-484d-8631-a3b158b012a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:31.838635 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9cb2-6a31-484d-8631-a3b158b012a2 None admin] PATH_INFO: `/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:31.839882 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa9cb2-6a31-484d-8631-a3b158b012a2 None admin] RBAC: Authorizing `identity:get_group(group_id=4bdb1f914c0b4f7f99e863084f9e1806)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:31.841333 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa9cb2-6a31-484d-8631-a3b158b012a2 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:31.843191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 195/379] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:31 2022] GET /identity/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:31.843761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa9cb2-6a31-484d-8631-a3b158b012a2 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:31.865803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f24031-b40b-432b-8bbd-138fa5b3d211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:31.867091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f24031-b40b-432b-8bbd-138fa5b3d211 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:31.867673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f24031-b40b-432b-8bbd-138fa5b3d211 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:31.868177 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f24031-b40b-432b-8bbd-138fa5b3d211 None admin] PATH_INFO: `/v3/groups/fbb817a38c174573bffabbb90fd1f23a` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:31.875277 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f24031-b40b-432b-8bbd-138fa5b3d211 None admin] RBAC: Authorizing `identity:get_group(group_id=fbb817a38c174573bffabbb90fd1f23a)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:31.877648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f24031-b40b-432b-8bbd-138fa5b3d211 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:31.881229 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 185/380] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:31 2022] GET /identity/v3/groups/fbb817a38c174573bffabbb90fd1f23a => generated 229 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:31.883276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f24031-b40b-432b-8bbd-138fa5b3d211 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:31.918198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:31.965794 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa9cb2-6a31-484d-8631-a3b158b012a2 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:31.996909 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f24031-b40b-432b-8bbd-138fa5b3d211 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:32.004904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ff251-6a3c-4c58-8c4c-250f2e771faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:32.005881 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ff251-6a3c-4c58-8c4c-250f2e771faf None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:32.006100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ff251-6a3c-4c58-8c4c-250f2e771faf None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:32.006309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ff251-6a3c-4c58-8c4c-250f2e771faf None admin] PATH_INFO: `/v3/roles/member` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:32.018396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ff251-6a3c-4c58-8c4c-250f2e771faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-c70ff251-6a3c-4c58-8c4c-250f2e771faf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:32.027384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:32.028283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:32.029669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:32.029669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:32.029669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:32.029669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:32.029669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:32.029669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:32.029669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:32.029669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:32.029669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:32.029669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:32.029669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:32.029669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:32.029669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 196/381] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:32.030379 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ff251-6a3c-4c58-8c4c-250f2e771faf None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:32.042476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b047a7c-a35b-47f2-808e-073e533f2b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:32.043509 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b047a7c-a35b-47f2-808e-073e533f2b3c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:32.043997 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b047a7c-a35b-47f2-808e-073e533f2b3c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:32.044490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b047a7c-a35b-47f2-808e-073e533f2b3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:32.045528 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b047a7c-a35b-47f2-808e-073e533f2b3c None admin] RBAC: Adding query filter params (group.id=4bdb1f914c0b4f7f99e863084f9e1806, role.id=5c2125e2125d40bca6590d50afd677d1, scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:32.046045 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b047a7c-a35b-47f2-808e-073e533f2b3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:32.047704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b047a7c-a35b-47f2-808e-073e533f2b3c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:32.054345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 186/382] 10.222.0.52 () {56 vars in 1376 bytes} [Thu May 19 13:40:31 2022] GET /identity/v3/role_assignments?group.id=4bdb1f914c0b4f7f99e863084f9e1806&scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea&role.id=5c2125e2125d40bca6590d50afd677d1 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:32.055612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b047a7c-a35b-47f2-808e-073e533f2b3c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:32.076512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:32.152149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ff251-6a3c-4c58-8c4c-250f2e771faf None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:32.173367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b047a7c-a35b-47f2-808e-073e533f2b3c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:32.174759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:32.194057 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6716c-31bf-4ba8-a008-c711bc6e74b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:32.194725 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6716c-31bf-4ba8-a008-c711bc6e74b8 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:32.195070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6716c-31bf-4ba8-a008-c711bc6e74b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:32.195367 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6716c-31bf-4ba8-a008-c711bc6e74b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:32.196159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6716c-31bf-4ba8-a008-c711bc6e74b8 None admin] RBAC: Adding query filter params (group.id=fbb817a38c174573bffabbb90fd1f23a, role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=2f0d02746fbd4bebabb402313125d4b8) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:32.196477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6716c-31bf-4ba8-a008-c711bc6e74b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:32.198076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6716c-31bf-4ba8-a008-c711bc6e74b8 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:32.204811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 197/383] 10.222.0.52 () {56 vars in 1376 bytes} [Thu May 19 13:40:32 2022] GET /identity/v3/role_assignments?group.id=fbb817a38c174573bffabbb90fd1f23a&scope.project.id=2f0d02746fbd4bebabb402313125d4b8&role.id=0893bb944dd94634accd3906aa391cd2 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:32.219227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1eaf0-2209-4f7e-a0b5-5f7d0573fedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:32.220299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1eaf0-2209-4f7e-a0b5-5f7d0573fedf None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:32.220795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1eaf0-2209-4f7e-a0b5-5f7d0573fedf None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:32.221303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1eaf0-2209-4f7e-a0b5-5f7d0573fedf None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:32.222338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1eaf0-2209-4f7e-a0b5-5f7d0573fedf None admin] RBAC: Adding query filter params (name=member) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:32.223047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1eaf0-2209-4f7e-a0b5-5f7d0573fedf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:32.225323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1eaf0-2209-4f7e-a0b5-5f7d0573fedf None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:32.236210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 187/384] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:32.242735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6716c-31bf-4ba8-a008-c711bc6e74b8 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:32.393349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6716c-31bf-4ba8-a008-c711bc6e74b8 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:32.434356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61949d-abd9-402b-82fb-d2e72649ffc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:32.435262 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61949d-abd9-402b-82fb-d2e72649ffc9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:32.435611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61949d-abd9-402b-82fb-d2e72649ffc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:32.435892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61949d-abd9-402b-82fb-d2e72649ffc9 None admin] PATH_INFO: `/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:32.437090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61949d-abd9-402b-82fb-d2e72649ffc9 None admin] RBAC: Authorizing `identity:get_group(group_id=4bdb1f914c0b4f7f99e863084f9e1806)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:32.438545 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61949d-abd9-402b-82fb-d2e72649ffc9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:32.440517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 198/385] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:32 2022] GET /identity/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:32.446287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1eaf0-2209-4f7e-a0b5-5f7d0573fedf None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:32.543300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:32.567645 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1eaf0-2209-4f7e-a0b5-5f7d0573fedf None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:32.605206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249a5a8-9dca-4cfb-aa83-eb9133226a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:32.606452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249a5a8-9dca-4cfb-aa83-eb9133226a03 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:32.606960 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249a5a8-9dca-4cfb-aa83-eb9133226a03 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:32.607453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249a5a8-9dca-4cfb-aa83-eb9133226a03 None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:32.608853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249a5a8-9dca-4cfb-aa83-eb9133226a03 None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:32.610510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249a5a8-9dca-4cfb-aa83-eb9133226a03 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:32.612840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 188/386] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:32 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:32.618734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61949d-abd9-402b-82fb-d2e72649ffc9 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:32.709605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:32.771259 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61949d-abd9-402b-82fb-d2e72649ffc9 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:32.804935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1fdf2-c068-4120-857c-f02aadf550c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:32.805816 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1fdf2-c068-4120-857c-f02aadf550c3 None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:32.806095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1fdf2-c068-4120-857c-f02aadf550c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:32.806367 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1fdf2-c068-4120-857c-f02aadf550c3 None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93/groups/4bdb1f914c0b4f7f99e863084f9e1806/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:32.808349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1fdf2-c068-4120-857c-f02aadf550c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9fbb046f448445799c63c2b7d78bd93, group_id=4bdb1f914c0b4f7f99e863084f9e1806, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:32.809768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1fdf2-c068-4120-857c-f02aadf550c3 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:32.826165 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 199/387] 10.222.0.52 () {58 vars in 1442 bytes} [Thu May 19 13:40:32 2022] PUT /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93/groups/4bdb1f914c0b4f7f99e863084f9e1806/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:40:33.739528 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49385f91-25ca-48e7-bcf2-9f80e198bf1f None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:33.740496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49385f91-25ca-48e7-bcf2-9f80e198bf1f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:33.740989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49385f91-25ca-48e7-bcf2-9f80e198bf1f None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:33.742589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 189/388] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:33.748651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3986cf7-3bd7-492e-bfbf-06df87115652 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:33.749085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3986cf7-3bd7-492e-bfbf-06df87115652 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:33.749200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3986cf7-3bd7-492e-bfbf-06df87115652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:33.772468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-d3986cf7-3bd7-492e-bfbf-06df87115652 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:33.774003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d3986cf7-3bd7-492e-bfbf-06df87115652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:33.843329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 200/389] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:33.881375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49385f91-25ca-48e7-bcf2-9f80e198bf1f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:34.017320 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49385f91-25ca-48e7-bcf2-9f80e198bf1f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:34.065458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea98f6-08c8-42a6-a8d2-3db985258c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:34.067124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea98f6-08c8-42a6-a8d2-3db985258c7c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:34.067234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea98f6-08c8-42a6-a8d2-3db985258c7c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:34.067476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea98f6-08c8-42a6-a8d2-3db985258c7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:34.082361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea98f6-08c8-42a6-a8d2-3db985258c7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-d0ea98f6-08c8-42a6-a8d2-3db985258c7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:34.090670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:34.091578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:34.092413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:34.092413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:34.092413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:34.092413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:34.092413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:34.092413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:34.092413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:34.092413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:34.092413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:34.092413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:34.092413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:34.092413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:34.092413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 190/390] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:40:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:34.099143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3986cf7-3bd7-492e-bfbf-06df87115652 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:34.161037 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:34.258621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3986cf7-3bd7-492e-bfbf-06df87115652 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:34.316609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b43d32-22f8-45ab-a6f8-08ff2ad52e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:34.317290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b43d32-22f8-45ab-a6f8-08ff2ad52e84 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:34.317578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b43d32-22f8-45ab-a6f8-08ff2ad52e84 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:34.317857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b43d32-22f8-45ab-a6f8-08ff2ad52e84 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:34.318623 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b43d32-22f8-45ab-a6f8-08ff2ad52e84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:34.318953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b43d32-22f8-45ab-a6f8-08ff2ad52e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:34.320332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b43d32-22f8-45ab-a6f8-08ff2ad52e84 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:34.334698 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 201/391] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:34.340252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea98f6-08c8-42a6-a8d2-3db985258c7c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:34.350608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:34.440687 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf79dd6-23b8-4fa0-841e-2bc89ad6c07c None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:34.441027 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf79dd6-23b8-4fa0-841e-2bc89ad6c07c None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:34.441433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf79dd6-23b8-4fa0-841e-2bc89ad6c07c None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:34.442846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 202/392] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:34.453246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea98f6-08c8-42a6-a8d2-3db985258c7c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:34.459614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286373d4-09fa-419a-b670-5f9b28de2d87 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:34.459905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286373d4-09fa-419a-b670-5f9b28de2d87 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:34.460190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286373d4-09fa-419a-b670-5f9b28de2d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:34.491980 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-286373d4-09fa-419a-b670-5f9b28de2d87 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:34.493196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-286373d4-09fa-419a-b670-5f9b28de2d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:34.516252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9379178-ba74-42b7-9d97-6823e7e90fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:34.517125 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9379178-ba74-42b7-9d97-6823e7e90fd7 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:34.517413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9379178-ba74-42b7-9d97-6823e7e90fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:34.517691 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9379178-ba74-42b7-9d97-6823e7e90fd7 None admin] PATH_INFO: `/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:34.519754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9379178-ba74-42b7-9d97-6823e7e90fd7 None admin] RBAC: Authorizing `identity:get_group(group_id=4bdb1f914c0b4f7f99e863084f9e1806)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:34.521405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9379178-ba74-42b7-9d97-6823e7e90fd7 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:34.523274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 191/393] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:34 2022] GET /identity/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:34.529655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9379178-ba74-42b7-9d97-6823e7e90fd7 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:34.637101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 203/394] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:34.658084 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286373d4-09fa-419a-b670-5f9b28de2d87 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:34.693923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:34.729504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:34.747945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9379178-ba74-42b7-9d97-6823e7e90fd7 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:34.811980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea4fff-eeee-444a-938c-95ae9104d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:34.812845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea4fff-eeee-444a-938c-95ae9104d7ea None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:34.813132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea4fff-eeee-444a-938c-95ae9104d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:34.813409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea4fff-eeee-444a-938c-95ae9104d7ea None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:34.814619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea4fff-eeee-444a-938c-95ae9104d7ea None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:34.818263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea4fff-eeee-444a-938c-95ae9104d7ea None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:34.820885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 192/395] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:34 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:34.826412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c267e6-9a38-42f5-a1f4-e8776ed11c94 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:34.826412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c267e6-9a38-42f5-a1f4-e8776ed11c94 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:34.826412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c267e6-9a38-42f5-a1f4-e8776ed11c94 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:34.826412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 193/396] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:34.827566 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c267e6-9a38-42f5-a1f4-e8776ed11c94 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:34.833922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286373d4-09fa-419a-b670-5f9b28de2d87 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:34.872782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8e36d-fa0d-4636-aa47-573d8bdf6e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:34.873841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8e36d-fa0d-4636-aa47-573d8bdf6e92 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:34.874263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8e36d-fa0d-4636-aa47-573d8bdf6e92 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:34.874699 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8e36d-fa0d-4636-aa47-573d8bdf6e92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:34.887797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8e36d-fa0d-4636-aa47-573d8bdf6e92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-4ea8e36d-fa0d-4636-aa47-573d8bdf6e92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:34.896109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:34.897122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:34.897998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:34.897998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:34.897998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:34.897998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:34.897998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:34.897998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:34.897998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:34.897998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:34.897998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:34.897998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:34.897998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:34.897998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:34.905181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 204/397] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:40:34.907108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81bf12-29cd-422c-80e5-8c8f165c8245 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:34.907585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81bf12-29cd-422c-80e5-8c8f165c8245 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:34.908036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81bf12-29cd-422c-80e5-8c8f165c8245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:34.934505 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-9b81bf12-29cd-422c-80e5-8c8f165c8245 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:34.937110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9b81bf12-29cd-422c-80e5-8c8f165c8245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:34.968288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c267e6-9a38-42f5-a1f4-e8776ed11c94 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:35.006273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c86008-a14c-49b4-96c5-8bdceab4c51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:35.007468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c86008-a14c-49b4-96c5-8bdceab4c51d None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:35.007925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c86008-a14c-49b4-96c5-8bdceab4c51d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:35.008449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c86008-a14c-49b4-96c5-8bdceab4c51d None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93/groups/4bdb1f914c0b4f7f99e863084f9e1806/roles/5c2125e2125d40bca6590d50afd677d1` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:35.010983 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c86008-a14c-49b4-96c5-8bdceab4c51d None admin] RBAC: Authorizing `identity:create_grant(project_id=d9fbb046f448445799c63c2b7d78bd93, group_id=4bdb1f914c0b4f7f99e863084f9e1806, role_id=5c2125e2125d40bca6590d50afd677d1)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:35.012767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c86008-a14c-49b4-96c5-8bdceab4c51d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:35.026905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 205/398] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:35.027860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81bf12-29cd-422c-80e5-8c8f165c8245 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:35.035684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 194/399] 10.222.0.52 () {58 vars in 1442 bytes} [Thu May 19 13:40:34 2022] PUT /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93/groups/4bdb1f914c0b4f7f99e863084f9e1806/roles/5c2125e2125d40bca6590d50afd677d1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:40:35.049749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c86008-a14c-49b4-96c5-8bdceab4c51d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:35.053571 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:35.090796 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:35.183160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c86008-a14c-49b4-96c5-8bdceab4c51d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:35.201000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81bf12-29cd-422c-80e5-8c8f165c8245 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:35.223078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa52f4-50a4-44de-a2f7-433ad9ed24c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:35.224056 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa52f4-50a4-44de-a2f7-433ad9ed24c8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:35.224350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa52f4-50a4-44de-a2f7-433ad9ed24c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:35.224629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa52f4-50a4-44de-a2f7-433ad9ed24c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:35.237509 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa52f4-50a4-44de-a2f7-433ad9ed24c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:35.245709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397d387-99fa-4c2b-ba7f-a358a0de06e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:35.246806 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397d387-99fa-4c2b-ba7f-a358a0de06e6 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-07fa52f4-50a4-44de-a2f7-433ad9ed24c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:35.247301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:35.248341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:35.249154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:35.249154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:35.249154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:35.249154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:35.249154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:35.249154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:35.249154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:35.249154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:35.249154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:35.249154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:35.249154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:35.249154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:35.249757 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397d387-99fa-4c2b-ba7f-a358a0de06e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:35.249757 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397d387-99fa-4c2b-ba7f-a358a0de06e6 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:35.249757 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397d387-99fa-4c2b-ba7f-a358a0de06e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:35.249757 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397d387-99fa-4c2b-ba7f-a358a0de06e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:35.251399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397d387-99fa-4c2b-ba7f-a358a0de06e6 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:35.253477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 195/400] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:40:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:40:35.255790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5401362-1f94-43c9-a61f-047c3fda3bc7 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:35.256076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5401362-1f94-43c9-a61f-047c3fda3bc7 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:35.256354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5401362-1f94-43c9-a61f-047c3fda3bc7 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:35.258239 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 196/401] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:35.258878 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5401362-1f94-43c9-a61f-047c3fda3bc7 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:35.265885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 206/402] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:35.268167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a871dee-29a4-41c9-bc03-be7544eb9b73 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:35.268458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a871dee-29a4-41c9-bc03-be7544eb9b73 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:35.268739 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a871dee-29a4-41c9-bc03-be7544eb9b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:35.287903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-9a871dee-29a4-41c9-bc03-be7544eb9b73 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:35.289112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9a871dee-29a4-41c9-bc03-be7544eb9b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:35.340694 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:35.381475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 207/403] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:35.382447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a871dee-29a4-41c9-bc03-be7544eb9b73 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:35.387675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5401362-1f94-43c9-a61f-047c3fda3bc7 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:35.403996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:35.426223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b492124-282a-45f5-9528-286c9ce0cd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:35.427096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b492124-282a-45f5-9528-286c9ce0cd87 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:35.427496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b492124-282a-45f5-9528-286c9ce0cd87 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:35.427840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b492124-282a-45f5-9528-286c9ce0cd87 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:35.428665 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b492124-282a-45f5-9528-286c9ce0cd87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:35.429040 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b492124-282a-45f5-9528-286c9ce0cd87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:35.430568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b492124-282a-45f5-9528-286c9ce0cd87 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:35.440234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 197/404] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:35.441473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b492124-282a-45f5-9528-286c9ce0cd87 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:35.516066 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a871dee-29a4-41c9-bc03-be7544eb9b73 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:35.556646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85daaf2-4391-45dc-9df2-50d7fe89b23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:35.557254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b492124-282a-45f5-9528-286c9ce0cd87 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:35.557508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85daaf2-4391-45dc-9df2-50d7fe89b23f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:35.557794 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85daaf2-4391-45dc-9df2-50d7fe89b23f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:35.558071 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85daaf2-4391-45dc-9df2-50d7fe89b23f None admin] PATH_INFO: `/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:35.559383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85daaf2-4391-45dc-9df2-50d7fe89b23f None admin] RBAC: Authorizing `identity:get_group(group_id=4bdb1f914c0b4f7f99e863084f9e1806)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:35.560920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85daaf2-4391-45dc-9df2-50d7fe89b23f None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:35.562946 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 208/405] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:35 2022] GET /identity/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:35.563729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85daaf2-4391-45dc-9df2-50d7fe89b23f None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:35.583143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:35.604099 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e34cd-fd9e-4bf7-ae5e-4ca69fe0c596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:35.605049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e34cd-fd9e-4bf7-ae5e-4ca69fe0c596 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:35.605436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e34cd-fd9e-4bf7-ae5e-4ca69fe0c596 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:35.605871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e34cd-fd9e-4bf7-ae5e-4ca69fe0c596 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:35.617159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e34cd-fd9e-4bf7-ae5e-4ca69fe0c596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-276e34cd-fd9e-4bf7-ae5e-4ca69fe0c596 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:35.625361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:35.627144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:35.629521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:35.629521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:35.629521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:35.629521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:35.629521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:35.629521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:35.629521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:35.629521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:35.629521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:35.629521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:35.629521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:35.629521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:35.629521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 198/406] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:40:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:35.631400 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e34cd-fd9e-4bf7-ae5e-4ca69fe0c596 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:35.657464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:35.674038 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85daaf2-4391-45dc-9df2-50d7fe89b23f None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:35.715917 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db6044-6f26-42ca-a5c0-2eaa59a7bdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:35.716708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db6044-6f26-42ca-a5c0-2eaa59a7bdbc None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:35.716995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db6044-6f26-42ca-a5c0-2eaa59a7bdbc None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:35.717275 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db6044-6f26-42ca-a5c0-2eaa59a7bdbc None admin] PATH_INFO: `/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:35.718534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db6044-6f26-42ca-a5c0-2eaa59a7bdbc None admin] RBAC: Authorizing `identity:get_group(group_id=4bdb1f914c0b4f7f99e863084f9e1806)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:35.720295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db6044-6f26-42ca-a5c0-2eaa59a7bdbc None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:35.722373 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 209/407] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:35 2022] GET /identity/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:35.723341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db6044-6f26-42ca-a5c0-2eaa59a7bdbc None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:35.770737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e34cd-fd9e-4bf7-ae5e-4ca69fe0c596 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:35.818257 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dd9c7-87b9-41d6-a133-a3da17d06307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:35.819160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dd9c7-87b9-41d6-a133-a3da17d06307 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:35.819699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dd9c7-87b9-41d6-a133-a3da17d06307 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:35.819984 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dd9c7-87b9-41d6-a133-a3da17d06307 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:35.821581 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8dd9c7-87b9-41d6-a133-a3da17d06307 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:35.823144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8dd9c7-87b9-41d6-a133-a3da17d06307 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:35.826026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 199/408] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:35 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:35.827409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dd9c7-87b9-41d6-a133-a3da17d06307 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:35.837242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db6044-6f26-42ca-a5c0-2eaa59a7bdbc None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:35.875598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37877583-23f8-4059-be78-aaa66614a623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:35.876452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37877583-23f8-4059-be78-aaa66614a623 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:35.876839 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37877583-23f8-4059-be78-aaa66614a623 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:35.877227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37877583-23f8-4059-be78-aaa66614a623 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:35.878148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37877583-23f8-4059-be78-aaa66614a623 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:35.878553 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37877583-23f8-4059-be78-aaa66614a623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:35.880894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37877583-23f8-4059-be78-aaa66614a623 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:35.895423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 210/409] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:35.896344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:35.897165 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37877583-23f8-4059-be78-aaa66614a623 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:35.917000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:35.953001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dd9c7-87b9-41d6-a133-a3da17d06307 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:35.991433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37125ed4-d23c-4d7e-ad2d-1b3db83309a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:35.992221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37125ed4-d23c-4d7e-ad2d-1b3db83309a8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:35.992505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37125ed4-d23c-4d7e-ad2d-1b3db83309a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:35.992781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37125ed4-d23c-4d7e-ad2d-1b3db83309a8 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:35.994035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37125ed4-d23c-4d7e-ad2d-1b3db83309a8 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:35.995545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37125ed4-d23c-4d7e-ad2d-1b3db83309a8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:35.997514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 200/410] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:35 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:35.998275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37125ed4-d23c-4d7e-ad2d-1b3db83309a8 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:36.015364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37877583-23f8-4059-be78-aaa66614a623 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:36.054728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5db6f-607a-4cba-8957-6a867ebcddee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:36.055839 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5db6f-607a-4cba-8957-6a867ebcddee None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:36.056283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5db6f-607a-4cba-8957-6a867ebcddee None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:36.056713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5db6f-607a-4cba-8957-6a867ebcddee None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea/groups/4bdb1f914c0b4f7f99e863084f9e1806/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:36.059084 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5db6f-607a-4cba-8957-6a867ebcddee None admin] RBAC: Authorizing `identity:create_grant(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea, group_id=4bdb1f914c0b4f7f99e863084f9e1806, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:36.060894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5db6f-607a-4cba-8957-6a867ebcddee None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:36.082885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 211/411] 10.222.0.52 () {58 vars in 1442 bytes} [Thu May 19 13:40:35 2022] PUT /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea/groups/4bdb1f914c0b4f7f99e863084f9e1806/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:40:36.082885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb2907-611c-462e-be71-fa80378ad94c None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:36.082885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb2907-611c-462e-be71-fa80378ad94c None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:36.082885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb2907-611c-462e-be71-fa80378ad94c None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:36.082885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 212/412] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:36.082885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb2907-611c-462e-be71-fa80378ad94c None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:36.120334 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37125ed4-d23c-4d7e-ad2d-1b3db83309a8 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:36.144044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:36.162119 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd4e10-6562-4e3d-8200-cfaa4e37ae7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:36.163189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd4e10-6562-4e3d-8200-cfaa4e37ae7e None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:36.163558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd4e10-6562-4e3d-8200-cfaa4e37ae7e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:36.163903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd4e10-6562-4e3d-8200-cfaa4e37ae7e None admin] PATH_INFO: `/v3/groups/fbb817a38c174573bffabbb90fd1f23a` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:36.165211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd4e10-6562-4e3d-8200-cfaa4e37ae7e None admin] RBAC: Authorizing `identity:get_group(group_id=fbb817a38c174573bffabbb90fd1f23a)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:36.166972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd4e10-6562-4e3d-8200-cfaa4e37ae7e None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:36.173630 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 201/413] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:35 2022] GET /identity/v3/groups/fbb817a38c174573bffabbb90fd1f23a => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:36.175227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a172d09-5b25-4e29-9b92-7f6f77187aaf None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:36.176114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a172d09-5b25-4e29-9b92-7f6f77187aaf None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:36.177045 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a172d09-5b25-4e29-9b92-7f6f77187aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:36.195988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3a172d09-5b25-4e29-9b92-7f6f77187aaf None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:36.197322 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3a172d09-5b25-4e29-9b92-7f6f77187aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:36.207496 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:36.248488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb2907-611c-462e-be71-fa80378ad94c None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:36.298485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1e68b-ce8b-468b-939f-5b31820e36ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:36.300108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1e68b-ce8b-468b-939f-5b31820e36ea None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:36.300547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1e68b-ce8b-468b-939f-5b31820e36ea None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:36.301574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1e68b-ce8b-468b-939f-5b31820e36ea None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea/groups/4bdb1f914c0b4f7f99e863084f9e1806/roles/5c2125e2125d40bca6590d50afd677d1` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:36.304203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1e68b-ce8b-468b-939f-5b31820e36ea None admin] RBAC: Authorizing `identity:create_grant(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea, group_id=4bdb1f914c0b4f7f99e863084f9e1806, role_id=5c2125e2125d40bca6590d50afd677d1)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:36.304387 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 202/414] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:36.305354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a172d09-5b25-4e29-9b92-7f6f77187aaf None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:36.306169 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1e68b-ce8b-468b-939f-5b31820e36ea None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:36.333920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 213/415] 10.222.0.52 () {58 vars in 1442 bytes} [Thu May 19 13:40:36 2022] PUT /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea/groups/4bdb1f914c0b4f7f99e863084f9e1806/roles/5c2125e2125d40bca6590d50afd677d1 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:40:36.334734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1e68b-ce8b-468b-939f-5b31820e36ea None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:36.424608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a172d09-5b25-4e29-9b92-7f6f77187aaf None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:36.445428 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:36.465124 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1e68b-ce8b-468b-939f-5b31820e36ea None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:36.465687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56b281-605d-46e5-9410-977847e3ceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:36.466828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56b281-605d-46e5-9410-977847e3ceb0 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:36.467328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56b281-605d-46e5-9410-977847e3ceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:36.467739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56b281-605d-46e5-9410-977847e3ceb0 None admin] PATH_INFO: `/v3/projects/2f0d02746fbd4bebabb402313125d4b8` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:36.469881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56b281-605d-46e5-9410-977847e3ceb0 None admin] RBAC: Authorizing `identity:get_project(project_id=2f0d02746fbd4bebabb402313125d4b8)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:36.471567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56b281-605d-46e5-9410-977847e3ceb0 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:36.473900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 203/416] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:36 2022] GET /identity/v3/projects/2f0d02746fbd4bebabb402313125d4b8 => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:36.486899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56b281-605d-46e5-9410-977847e3ceb0 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:36.522886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdee08c-8c9b-4ab4-9a3c-7fc9b01adffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:36.523645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdee08c-8c9b-4ab4-9a3c-7fc9b01adffa None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:36.523937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdee08c-8c9b-4ab4-9a3c-7fc9b01adffa None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:36.524221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdee08c-8c9b-4ab4-9a3c-7fc9b01adffa None admin] PATH_INFO: `/v3/roles/member` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:36.531750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:36.537926 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdee08c-8c9b-4ab4-9a3c-7fc9b01adffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-9fdee08c-8c9b-4ab4-9a3c-7fc9b01adffa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:36.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:36.557310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:36.558088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:36.558088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:36.558088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:36.558088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:36.558088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:36.558088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:36.558088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:36.558088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:36.558088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:36.558088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:36.558088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:36.558088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:36.558088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 214/417] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:36.563417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdee08c-8c9b-4ab4-9a3c-7fc9b01adffa None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:36.629549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56b281-605d-46e5-9410-977847e3ceb0 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:36.666266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c1e4e-7e9e-4373-a6bb-bc91f90e5341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:36.667175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c1e4e-7e9e-4373-a6bb-bc91f90e5341 None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:36.667455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c1e4e-7e9e-4373-a6bb-bc91f90e5341 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:36.667728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c1e4e-7e9e-4373-a6bb-bc91f90e5341 None admin] PATH_INFO: `/v3/projects/2f0d02746fbd4bebabb402313125d4b8/groups/fbb817a38c174573bffabbb90fd1f23a/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:36.669790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c1e4e-7e9e-4373-a6bb-bc91f90e5341 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f0d02746fbd4bebabb402313125d4b8, group_id=fbb817a38c174573bffabbb90fd1f23a, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:36.671262 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c1e4e-7e9e-4373-a6bb-bc91f90e5341 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:36.692645 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 204/418] 10.222.0.52 () {58 vars in 1442 bytes} [Thu May 19 13:40:36 2022] PUT /identity/v3/projects/2f0d02746fbd4bebabb402313125d4b8/groups/fbb817a38c174573bffabbb90fd1f23a/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:40:36.748006 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdee08c-8c9b-4ab4-9a3c-7fc9b01adffa None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:36.801544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b56ea2-756d-4bcc-acfa-77f6a7048754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:36.801544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b56ea2-756d-4bcc-acfa-77f6a7048754 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:36.801953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b56ea2-756d-4bcc-acfa-77f6a7048754 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:36.802222 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b56ea2-756d-4bcc-acfa-77f6a7048754 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:36.803016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b56ea2-756d-4bcc-acfa-77f6a7048754 None admin] RBAC: Adding query filter params (name=member) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:36.803329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b56ea2-756d-4bcc-acfa-77f6a7048754 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:36.804710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b56ea2-756d-4bcc-acfa-77f6a7048754 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:36.814235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 215/419] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:36.820102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c1e4e-7e9e-4373-a6bb-bc91f90e5341 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:36.945751 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:37.027032 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c1e4e-7e9e-4373-a6bb-bc91f90e5341 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:37.069423 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd347b1-073e-461e-96e7-0817f8b3c475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:37.070433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd347b1-073e-461e-96e7-0817f8b3c475 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:37.070968 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd347b1-073e-461e-96e7-0817f8b3c475 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:37.071469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd347b1-073e-461e-96e7-0817f8b3c475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:37.154116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd347b1-073e-461e-96e7-0817f8b3c475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:37.160749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd347b1-073e-461e-96e7-0817f8b3c475 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:37.218505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:37.277158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 205/420] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:37.282517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b56ea2-756d-4bcc-acfa-77f6a7048754 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:37.407235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:37.475643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b56ea2-756d-4bcc-acfa-77f6a7048754 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:37.532570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdd28c-c93d-4662-a5fb-88f9d2e1877f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:37.533442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdd28c-c93d-4662-a5fb-88f9d2e1877f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:37.533724 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdd28c-c93d-4662-a5fb-88f9d2e1877f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:37.534001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdd28c-c93d-4662-a5fb-88f9d2e1877f None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:37.535826 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acdd28c-c93d-4662-a5fb-88f9d2e1877f None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:37.537582 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acdd28c-c93d-4662-a5fb-88f9d2e1877f None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:37.539824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 216/421] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:37 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:37.545443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd347b1-073e-461e-96e7-0817f8b3c475 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:37.631400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5287c9-3ab4-4759-9a1c-182cf1b1cbb7 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:37.631973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5287c9-3ab4-4759-9a1c-182cf1b1cbb7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:37.632431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5287c9-3ab4-4759-9a1c-182cf1b1cbb7 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:37.633966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 217/422] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:40:37.645189 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decae83f-cec5-4818-9c4c-f82c0cecb461 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:37.645916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decae83f-cec5-4818-9c4c-f82c0cecb461 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:37.646371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decae83f-cec5-4818-9c4c-f82c0cecb461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:37.666123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-decae83f-cec5-4818-9c4c-f82c0cecb461 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:37.667635 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-decae83f-cec5-4818-9c4c-f82c0cecb461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:37.695216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd347b1-073e-461e-96e7-0817f8b3c475 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:37.764784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51ed31-bc97-48f1-8b18-7f5f4c5a3853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:37.765938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ed31-bc97-48f1-8b18-7f5f4c5a3853 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:37.766400 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ed31-bc97-48f1-8b18-7f5f4c5a3853 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:37.766850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ed31-bc97-48f1-8b18-7f5f4c5a3853 None admin] PATH_INFO: `/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:37.768362 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51ed31-bc97-48f1-8b18-7f5f4c5a3853 None admin] RBAC: Authorizing `identity:get_group(group_id=4bdb1f914c0b4f7f99e863084f9e1806)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:37.770124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51ed31-bc97-48f1-8b18-7f5f4c5a3853 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:37.779873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 206/423] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:37 2022] GET /identity/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:37.783455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51ed31-bc97-48f1-8b18-7f5f4c5a3853 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:37.795791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 218/424] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:37.809408 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:37.821762 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decae83f-cec5-4818-9c4c-f82c0cecb461 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:37.883241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:37.930688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decae83f-cec5-4818-9c4c-f82c0cecb461 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:37.950493 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51ed31-bc97-48f1-8b18-7f5f4c5a3853 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:37.970714 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c3135-329a-4d71-93b7-32599ca455e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:37.971851 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c3135-329a-4d71-93b7-32599ca455e9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:37.974397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c3135-329a-4d71-93b7-32599ca455e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:37.975565 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c3135-329a-4d71-93b7-32599ca455e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:37.997561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c3135-329a-4d71-93b7-32599ca455e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-7d8c3135-329a-4d71-93b7-32599ca455e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:38.009044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:38.009967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:38.010740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:38.010740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:38.010740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:38.010740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:38.010740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:38.010740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:38.010740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:38.010740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:38.010740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:38.010740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:38.010740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:38.010740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:38.019896 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 219/425] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:40:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:38.022541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c3135-329a-4d71-93b7-32599ca455e9 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:38.041345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d82e4e-8b41-45f2-89b2-c9c1bfc0c099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:38.042289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d82e4e-8b41-45f2-89b2-c9c1bfc0c099 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:38.042747 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d82e4e-8b41-45f2-89b2-c9c1bfc0c099 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:38.043240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d82e4e-8b41-45f2-89b2-c9c1bfc0c099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:38.044283 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d82e4e-8b41-45f2-89b2-c9c1bfc0c099 None admin] RBAC: Adding query filter params (group.id=4bdb1f914c0b4f7f99e863084f9e1806, role.id=a77e3ea4b46b4a3aaca8e814f53809b5, scope.project.id=d9fbb046f448445799c63c2b7d78bd93) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:38.044776 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d82e4e-8b41-45f2-89b2-c9c1bfc0c099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:38.046391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d82e4e-8b41-45f2-89b2-c9c1bfc0c099 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:38.059925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 207/426] 10.222.0.52 () {56 vars in 1376 bytes} [Thu May 19 13:40:37 2022] GET /identity/v3/role_assignments?group.id=4bdb1f914c0b4f7f99e863084f9e1806&scope.project.id=d9fbb046f448445799c63c2b7d78bd93&role.id=a77e3ea4b46b4a3aaca8e814f53809b5 => generated 611 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:38.192586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c3135-329a-4d71-93b7-32599ca455e9 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:38.208206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:38.237909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979eed6-2519-4dfe-986f-c0a390b720ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:38.237909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979eed6-2519-4dfe-986f-c0a390b720ec None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:38.237909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979eed6-2519-4dfe-986f-c0a390b720ec None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:38.237909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979eed6-2519-4dfe-986f-c0a390b720ec None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:38.238877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979eed6-2519-4dfe-986f-c0a390b720ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:38.238877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979eed6-2519-4dfe-986f-c0a390b720ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:38.238877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979eed6-2519-4dfe-986f-c0a390b720ec None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:38.252732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 220/427] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:38.260602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d82e4e-8b41-45f2-89b2-c9c1bfc0c099 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:38.465682 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:38.513100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d82e4e-8b41-45f2-89b2-c9c1bfc0c099 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:38.603382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ee3d2-e4ff-4790-8bb6-e7d77c6656be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:38.604067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ee3d2-e4ff-4790-8bb6-e7d77c6656be None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:38.604348 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ee3d2-e4ff-4790-8bb6-e7d77c6656be None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:38.604621 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ee3d2-e4ff-4790-8bb6-e7d77c6656be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:38.713783 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ee3d2-e4ff-4790-8bb6-e7d77c6656be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:38.716070 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ee3d2-e4ff-4790-8bb6-e7d77c6656be None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:38.810462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:38.830732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc4273-2e9e-4515-857f-0850bd4f6420 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:38.831249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc4273-2e9e-4515-857f-0850bd4f6420 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:38.831326 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc4273-2e9e-4515-857f-0850bd4f6420 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:38.833103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 221/428] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:38.837915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df17ca-bd9b-4e4f-a1e8-c85a10290667 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:38.838205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df17ca-bd9b-4e4f-a1e8-c85a10290667 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:38.838489 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df17ca-bd9b-4e4f-a1e8-c85a10290667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:38.853368 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 208/429] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:38.860219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ee3d2-e4ff-4790-8bb6-e7d77c6656be None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:38.866364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f2df17ca-bd9b-4e4f-a1e8-c85a10290667 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:38.867774 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f2df17ca-bd9b-4e4f-a1e8-c85a10290667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:38.948446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 222/430] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:38.969873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ee3d2-e4ff-4790-8bb6-e7d77c6656be None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:38.976056 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df17ca-bd9b-4e4f-a1e8-c85a10290667 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:39.013674 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a598eba-e9d5-4d67-bd06-342cd14c61ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:39.014537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a598eba-e9d5-4d67-bd06-342cd14c61ea None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:39.014819 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a598eba-e9d5-4d67-bd06-342cd14c61ea None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:39.015124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a598eba-e9d5-4d67-bd06-342cd14c61ea None admin] PATH_INFO: `/v3/projects/d9fbb046f448445799c63c2b7d78bd93` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:39.016313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a598eba-e9d5-4d67-bd06-342cd14c61ea None admin] RBAC: Authorizing `identity:get_project(project_id=d9fbb046f448445799c63c2b7d78bd93)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:39.017756 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a598eba-e9d5-4d67-bd06-342cd14c61ea None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:39.020332 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 209/431] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:38 2022] GET /identity/v3/projects/d9fbb046f448445799c63c2b7d78bd93 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:39.029420 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a598eba-e9d5-4d67-bd06-342cd14c61ea None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:39.089543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df17ca-bd9b-4e4f-a1e8-c85a10290667 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:39.098831 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:39.111728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:39.140591 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a905a6-3a74-4115-9930-009c724d122b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:39.141480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a905a6-3a74-4115-9930-009c724d122b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:39.141830 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a905a6-3a74-4115-9930-009c724d122b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:39.142174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a905a6-3a74-4115-9930-009c724d122b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:39.153137 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a905a6-3a74-4115-9930-009c724d122b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-f0a905a6-3a74-4115-9930-009c724d122b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:39.161321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:39.162230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:39.163450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:39.163450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:39.163450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:39.163450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:39.163450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:39.163450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:39.163450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:39.163450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:39.163450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:39.163450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:39.163450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:40:39.163450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:39.163450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 223/432] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:40:39.164109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88338248-a2af-4a92-b261-4e8927a9b25b None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:39.164109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88338248-a2af-4a92-b261-4e8927a9b25b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:39.164458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88338248-a2af-4a92-b261-4e8927a9b25b None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:39.165710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 224/433] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:39.170207 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88338248-a2af-4a92-b261-4e8927a9b25b None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:39.184496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a598eba-e9d5-4d67-bd06-342cd14c61ea None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:39.220940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe73ca-ba35-44df-8ad5-4af7aceb2975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:39.222077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe73ca-ba35-44df-8ad5-4af7aceb2975 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:39.222564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe73ca-ba35-44df-8ad5-4af7aceb2975 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:39.223004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe73ca-ba35-44df-8ad5-4af7aceb2975 None admin] PATH_INFO: `/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:39.224353 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe73ca-ba35-44df-8ad5-4af7aceb2975 None admin] RBAC: Authorizing `identity:get_group(group_id=4bdb1f914c0b4f7f99e863084f9e1806)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:39.226076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe73ca-ba35-44df-8ad5-4af7aceb2975 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:39.228323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 210/434] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:39 2022] GET /identity/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:39.229484 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9604524-a3c8-4e70-8a78-f40a940c2a9d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:39.229775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9604524-a3c8-4e70-8a78-f40a940c2a9d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:39.230156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9604524-a3c8-4e70-8a78-f40a940c2a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:39.247836 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-b9604524-a3c8-4e70-8a78-f40a940c2a9d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:39.249003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b9604524-a3c8-4e70-8a78-f40a940c2a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:39.283106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88338248-a2af-4a92-b261-4e8927a9b25b None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:39.317786 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 211/435] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:39.318629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9604524-a3c8-4e70-8a78-f40a940c2a9d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:39.323060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4c85f-53f0-4d99-9aa6-9d65ce0e558a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:39.323972 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4c85f-53f0-4d99-9aa6-9d65ce0e558a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:39.324475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4c85f-53f0-4d99-9aa6-9d65ce0e558a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:39.325012 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4c85f-53f0-4d99-9aa6-9d65ce0e558a None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:39.325842 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4c85f-53f0-4d99-9aa6-9d65ce0e558a None admin] RBAC: Adding query filter params (name=member) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:39.326239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4c85f-53f0-4d99-9aa6-9d65ce0e558a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:39.327726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4c85f-53f0-4d99-9aa6-9d65ce0e558a None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:39.341082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 225/436] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:40:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:39.346717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4c85f-53f0-4d99-9aa6-9d65ce0e558a None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:39.377494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:39.379288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:39.437419 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9604524-a3c8-4e70-8a78-f40a940c2a9d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:39.464199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4c85f-53f0-4d99-9aa6-9d65ce0e558a None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:39.472100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a33d6-e056-4a8e-a5a4-4e71d24c6974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:39.472709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a33d6-e056-4a8e-a5a4-4e71d24c6974 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:39.472985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a33d6-e056-4a8e-a5a4-4e71d24c6974 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:39.473346 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a33d6-e056-4a8e-a5a4-4e71d24c6974 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:39.474098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a33d6-e056-4a8e-a5a4-4e71d24c6974 None admin] RBAC: Adding query filter params (group.id=4bdb1f914c0b4f7f99e863084f9e1806, role.id=5c2125e2125d40bca6590d50afd677d1, scope.project.id=d9fbb046f448445799c63c2b7d78bd93) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:39.474407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a33d6-e056-4a8e-a5a4-4e71d24c6974 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:39.475879 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a33d6-e056-4a8e-a5a4-4e71d24c6974 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:39.486897 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 212/437] 10.222.0.52 () {56 vars in 1376 bytes} [Thu May 19 13:40:39 2022] GET /identity/v3/role_assignments?group.id=4bdb1f914c0b4f7f99e863084f9e1806&scope.project.id=d9fbb046f448445799c63c2b7d78bd93&role.id=5c2125e2125d40bca6590d50afd677d1 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:39.486897 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a33d6-e056-4a8e-a5a4-4e71d24c6974 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:39.529500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae3089-26ea-4856-b289-96251a261a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:39.530337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3089-26ea-4856-b289-96251a261a06 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:39.530622 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3089-26ea-4856-b289-96251a261a06 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:39.530928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3089-26ea-4856-b289-96251a261a06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:39.544390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae3089-26ea-4856-b289-96251a261a06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-a0ae3089-26ea-4856-b289-96251a261a06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:39.554115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:39.555109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:39.555901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:39.555901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:39.555901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:39.555901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:39.555901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:39.555901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:39.555901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:39.555901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:39.555901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:39.555901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:39.555901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:39.555901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:39.560332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 226/438] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:40:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:40:39.561299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae3089-26ea-4856-b289-96251a261a06 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:39.608568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a33d6-e056-4a8e-a5a4-4e71d24c6974 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:39.645786 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:39.652778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b496a2-4ffa-469d-a7de-614f6669ee4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:39.653975 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b496a2-4ffa-469d-a7de-614f6669ee4d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:39.654489 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b496a2-4ffa-469d-a7de-614f6669ee4d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:39.656047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b496a2-4ffa-469d-a7de-614f6669ee4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:39.674533 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:39.697697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae3089-26ea-4856-b289-96251a261a06 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:39.743817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70499d-e9d9-4b3e-bfa4-eb03a203a88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:39.744412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70499d-e9d9-4b3e-bfa4-eb03a203a88a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:39.744698 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70499d-e9d9-4b3e-bfa4-eb03a203a88a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:39.744974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70499d-e9d9-4b3e-bfa4-eb03a203a88a None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:39.745737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70499d-e9d9-4b3e-bfa4-eb03a203a88a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:39.746099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70499d-e9d9-4b3e-bfa4-eb03a203a88a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:39.747805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70499d-e9d9-4b3e-bfa4-eb03a203a88a None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:39.750714 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b496a2-4ffa-469d-a7de-614f6669ee4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:39.752452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b496a2-4ffa-469d-a7de-614f6669ee4d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:39.761429 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 227/439] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:39.762881 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70499d-e9d9-4b3e-bfa4-eb03a203a88a None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:39.881468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 213/440] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:39.883178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbce5f-db46-451f-914d-a7bfbb50c4c9 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:39.883612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbce5f-db46-451f-914d-a7bfbb50c4c9 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:39.884066 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbce5f-db46-451f-914d-a7bfbb50c4c9 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:39.885333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 214/441] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:39.888563 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbce5f-db46-451f-914d-a7bfbb50c4c9 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:39.896067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70499d-e9d9-4b3e-bfa4-eb03a203a88a None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:39.900795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:39.942361 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a3b90-84f6-4551-b280-df9a4f79b570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:39.943089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a3b90-84f6-4551-b280-df9a4f79b570 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:39.943384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a3b90-84f6-4551-b280-df9a4f79b570 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:39.943667 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a3b90-84f6-4551-b280-df9a4f79b570 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:39.964358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:40.006702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbce5f-db46-451f-914d-a7bfbb50c4c9 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:40.013640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a3b90-84f6-4551-b280-df9a4f79b570 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:40.015107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a3b90-84f6-4551-b280-df9a4f79b570 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:40.041032 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b2935-1897-4db0-9a59-4761f1daa9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:40.041878 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b2935-1897-4db0-9a59-4761f1daa9d5 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:40.042164 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b2935-1897-4db0-9a59-4761f1daa9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:40.042435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b2935-1897-4db0-9a59-4761f1daa9d5 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:40.043633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b2935-1897-4db0-9a59-4761f1daa9d5 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:40.045035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b2935-1897-4db0-9a59-4761f1daa9d5 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:40.046913 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 215/442] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:39 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:40.047841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909479f6-45cf-4a71-a84e-7dd7398a56ed None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:40.048271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909479f6-45cf-4a71-a84e-7dd7398a56ed None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:40.048782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909479f6-45cf-4a71-a84e-7dd7398a56ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:40.064685 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-909479f6-45cf-4a71-a84e-7dd7398a56ed None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:40.065848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-909479f6-45cf-4a71-a84e-7dd7398a56ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:40.117559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 228/443] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:40:40.118404 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a3b90-84f6-4551-b280-df9a4f79b570 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:40.135346 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 216/444] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:40.136475 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909479f6-45cf-4a71-a84e-7dd7398a56ed None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:40.144018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:40.186336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:40.236556 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909479f6-45cf-4a71-a84e-7dd7398a56ed None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:40.236810 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a3b90-84f6-4551-b280-df9a4f79b570 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:40.272056 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21e66d-c4a5-427b-b87d-f9ddcabbdb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:40.272646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778d2aa-99b3-4cee-bdd2-ded866292b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:40.272849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21e66d-c4a5-427b-b87d-f9ddcabbdb01 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:40.273120 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21e66d-c4a5-427b-b87d-f9ddcabbdb01 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:40.273406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21e66d-c4a5-427b-b87d-f9ddcabbdb01 None admin] PATH_INFO: `/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:40.273500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778d2aa-99b3-4cee-bdd2-ded866292b35 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:40.273779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778d2aa-99b3-4cee-bdd2-ded866292b35 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:40.274058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778d2aa-99b3-4cee-bdd2-ded866292b35 None admin] PATH_INFO: `/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:40.274600 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21e66d-c4a5-427b-b87d-f9ddcabbdb01 None admin] RBAC: Authorizing `identity:get_project(project_id=0e6385af96bc45469d8aaf5fb3e8d7ea)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:40.275257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778d2aa-99b3-4cee-bdd2-ded866292b35 None admin] RBAC: Authorizing `identity:get_group(group_id=4bdb1f914c0b4f7f99e863084f9e1806)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:40.276055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21e66d-c4a5-427b-b87d-f9ddcabbdb01 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:40.276673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778d2aa-99b3-4cee-bdd2-ded866292b35 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:40.278030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 217/445] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:40 2022] GET /identity/v3/projects/0e6385af96bc45469d8aaf5fb3e8d7ea => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:40.282935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 229/446] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:40 2022] GET /identity/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:40.283080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21e66d-c4a5-427b-b87d-f9ddcabbdb01 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:40.283447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778d2aa-99b3-4cee-bdd2-ded866292b35 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:40.356230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:40.381994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778d2aa-99b3-4cee-bdd2-ded866292b35 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:40.382187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21e66d-c4a5-427b-b87d-f9ddcabbdb01 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:40.401588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:40.417794 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075c460-2f26-43a8-9642-386d1d4f75e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:40.418586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075c460-2f26-43a8-9642-386d1d4f75e3 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:40.418886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075c460-2f26-43a8-9642-386d1d4f75e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:40.419165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075c460-2f26-43a8-9642-386d1d4f75e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:40.419442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531592e-82bc-4d75-afd6-aae2553579dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:40.420313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531592e-82bc-4d75-afd6-aae2553579dd None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:40.420595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531592e-82bc-4d75-afd6-aae2553579dd None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:40.420875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531592e-82bc-4d75-afd6-aae2553579dd None admin] PATH_INFO: `/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:40.422047 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531592e-82bc-4d75-afd6-aae2553579dd None admin] RBAC: Authorizing `identity:get_group(group_id=4bdb1f914c0b4f7f99e863084f9e1806)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:40.423516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531592e-82bc-4d75-afd6-aae2553579dd None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:40.425119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 230/447] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:40 2022] GET /identity/v3/groups/4bdb1f914c0b4f7f99e863084f9e1806 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:40.426523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531592e-82bc-4d75-afd6-aae2553579dd None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:40.434144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075c460-2f26-43a8-9642-386d1d4f75e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-6075c460-2f26-43a8-9642-386d1d4f75e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:40.441772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:40.442743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 218/448] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:40:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:40.443744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075c460-2f26-43a8-9642-386d1d4f75e3 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:40.528688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531592e-82bc-4d75-afd6-aae2553579dd None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:40.545187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075c460-2f26-43a8-9642-386d1d4f75e3 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:40.562643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbb77d-a860-4ae5-8a37-8b072d54e9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:40.563411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbb77d-a860-4ae5-8a37-8b072d54e9f0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:40.563702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbb77d-a860-4ae5-8a37-8b072d54e9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:40.563982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbb77d-a860-4ae5-8a37-8b072d54e9f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:40.564812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbb77d-a860-4ae5-8a37-8b072d54e9f0 None admin] RBAC: Adding query filter params (group.id=4bdb1f914c0b4f7f99e863084f9e1806, role.id=a77e3ea4b46b4a3aaca8e814f53809b5, scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:40.565129 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbb77d-a860-4ae5-8a37-8b072d54e9f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:40.566769 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbb77d-a860-4ae5-8a37-8b072d54e9f0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:40.572704 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 231/449] 10.222.0.52 () {56 vars in 1376 bytes} [Thu May 19 13:40:40 2022] GET /identity/v3/role_assignments?group.id=4bdb1f914c0b4f7f99e863084f9e1806&scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea&role.id=a77e3ea4b46b4a3aaca8e814f53809b5 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:40.573492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbb77d-a860-4ae5-8a37-8b072d54e9f0 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:40.590623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0421f96-119c-427a-bd5a-beca7a0e9b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:40.591341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0421f96-119c-427a-bd5a-beca7a0e9b3d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:40.591625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0421f96-119c-427a-bd5a-beca7a0e9b3d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:40.591895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0421f96-119c-427a-bd5a-beca7a0e9b3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:40.592745 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0421f96-119c-427a-bd5a-beca7a0e9b3d None admin] RBAC: Adding query filter params (group.id=4bdb1f914c0b4f7f99e863084f9e1806, role.id=5c2125e2125d40bca6590d50afd677d1, scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:40.593056 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0421f96-119c-427a-bd5a-beca7a0e9b3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:40.594542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0421f96-119c-427a-bd5a-beca7a0e9b3d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:40.596843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:40.615648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 219/450] 10.222.0.52 () {56 vars in 1376 bytes} [Thu May 19 13:40:40 2022] GET /identity/v3/role_assignments?group.id=4bdb1f914c0b4f7f99e863084f9e1806&scope.project.id=0e6385af96bc45469d8aaf5fb3e8d7ea&role.id=5c2125e2125d40bca6590d50afd677d1 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:40.646730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:40.708183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbb77d-a860-4ae5-8a37-8b072d54e9f0 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:40.748163 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbc697-2c8e-4dd0-8602-1d6493774f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:40.748861 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbc697-2c8e-4dd0-8602-1d6493774f1b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:40.749146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbc697-2c8e-4dd0-8602-1d6493774f1b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:40.749429 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbc697-2c8e-4dd0-8602-1d6493774f1b None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:40.750211 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbc697-2c8e-4dd0-8602-1d6493774f1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:40.750523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbc697-2c8e-4dd0-8602-1d6493774f1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:40.752012 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dbc697-2c8e-4dd0-8602-1d6493774f1b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:40.762323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 232/451] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:40.768201 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0421f96-119c-427a-bd5a-beca7a0e9b3d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:40.884473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0421f96-119c-427a-bd5a-beca7a0e9b3d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:40.919557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e5b8d-afb5-4134-a80e-9bfce1c1d6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:40.920208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e5b8d-afb5-4134-a80e-9bfce1c1d6db None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:40.920504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e5b8d-afb5-4134-a80e-9bfce1c1d6db None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:40.920808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e5b8d-afb5-4134-a80e-9bfce1c1d6db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:40.987379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:40.991052 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e5b8d-afb5-4134-a80e-9bfce1c1d6db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:40.993047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e5b8d-afb5-4134-a80e-9bfce1c1d6db None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:41.091441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 220/452] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:41.097499 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbc697-2c8e-4dd0-8602-1d6493774f1b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:41.198457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbc697-2c8e-4dd0-8602-1d6493774f1b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:41.210938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:41.232152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940ba89-094e-40ca-8619-0dab76bb9094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:41.233076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940ba89-094e-40ca-8619-0dab76bb9094 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:41.233363 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940ba89-094e-40ca-8619-0dab76bb9094 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:41.233641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940ba89-094e-40ca-8619-0dab76bb9094 None admin] PATH_INFO: `/v3/projects/2f0d02746fbd4bebabb402313125d4b8` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:41.234812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940ba89-094e-40ca-8619-0dab76bb9094 None admin] RBAC: Authorizing `identity:get_project(project_id=2f0d02746fbd4bebabb402313125d4b8)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:41.236264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940ba89-094e-40ca-8619-0dab76bb9094 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:41.237987 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 233/453] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:40:41 2022] GET /identity/v3/projects/2f0d02746fbd4bebabb402313125d4b8 => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:41.243759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e5b8d-afb5-4134-a80e-9bfce1c1d6db None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:41.338016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:41.341724 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e5b8d-afb5-4134-a80e-9bfce1c1d6db None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:41.375399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc626f-5ab2-4c95-ba26-ff4c8c09512c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:41.376232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc626f-5ab2-4c95-ba26-ff4c8c09512c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:41.376518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc626f-5ab2-4c95-ba26-ff4c8c09512c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:41.376794 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc626f-5ab2-4c95-ba26-ff4c8c09512c None admin] PATH_INFO: `/v3/groups/fbb817a38c174573bffabbb90fd1f23a` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:41.377947 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc626f-5ab2-4c95-ba26-ff4c8c09512c None admin] RBAC: Authorizing `identity:get_group(group_id=fbb817a38c174573bffabbb90fd1f23a)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:41.379575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc626f-5ab2-4c95-ba26-ff4c8c09512c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:41.381176 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 221/454] 10.222.0.52 () {56 vars in 1180 bytes} [Thu May 19 13:40:41 2022] GET /identity/v3/groups/fbb817a38c174573bffabbb90fd1f23a => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:41.386816 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940ba89-094e-40ca-8619-0dab76bb9094 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:41.483280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940ba89-094e-40ca-8619-0dab76bb9094 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:41.515238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fcd1c-445b-4270-a9aa-56c1c30fcf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:41.515883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fcd1c-445b-4270-a9aa-56c1c30fcf35 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:41.516167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fcd1c-445b-4270-a9aa-56c1c30fcf35 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:41.516445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fcd1c-445b-4270-a9aa-56c1c30fcf35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:41.517266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fcd1c-445b-4270-a9aa-56c1c30fcf35 None admin] RBAC: Adding query filter params (group.id=fbb817a38c174573bffabbb90fd1f23a, role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=2f0d02746fbd4bebabb402313125d4b8) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:41.517578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fcd1c-445b-4270-a9aa-56c1c30fcf35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:41.519034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fcd1c-445b-4270-a9aa-56c1c30fcf35 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:41.525393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 234/455] 10.222.0.52 () {56 vars in 1376 bytes} [Thu May 19 13:40:41 2022] GET /identity/v3/role_assignments?group.id=fbb817a38c174573bffabbb90fd1f23a&scope.project.id=2f0d02746fbd4bebabb402313125d4b8&role.id=0893bb944dd94634accd3906aa391cd2 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:44.207330 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c157ecc-df69-4739-a2e3-ff88df885fa6 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:44.208496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c157ecc-df69-4739-a2e3-ff88df885fa6 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:44.208496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c157ecc-df69-4739-a2e3-ff88df885fa6 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:44.209676 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 222/456] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:44.215309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2e70d-f881-409f-a95f-cb51dbb5f02a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:44.216141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2e70d-f881-409f-a95f-cb51dbb5f02a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:44.216718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2e70d-f881-409f-a95f-cb51dbb5f02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:44.244686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-9ff2e70d-f881-409f-a95f-cb51dbb5f02a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:44.247561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9ff2e70d-f881-409f-a95f-cb51dbb5f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:44.303861 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:44.340598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 235/457] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:44.382085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c157ecc-df69-4739-a2e3-ff88df885fa6 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:44.548845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c157ecc-df69-4739-a2e3-ff88df885fa6 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:44.598895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b743f23-639e-4713-955d-b359ed8ef9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:44.599729 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743f23-639e-4713-955d-b359ed8ef9cc None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:44.600021 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743f23-639e-4713-955d-b359ed8ef9cc None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:44.600302 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743f23-639e-4713-955d-b359ed8ef9cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:44.614971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b743f23-639e-4713-955d-b359ed8ef9cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:44.616427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b743f23-639e-4713-955d-b359ed8ef9cc None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:44.619833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423a103-0f1d-45ad-8c8b-f70249f70c66 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:44.620452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423a103-0f1d-45ad-8c8b-f70249f70c66 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:44.620949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423a103-0f1d-45ad-8c8b-f70249f70c66 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:44.623112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 236/458] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:40:44.628809 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b9c25-d196-4000-818f-f3a11d173c03 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:44.629931 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b9c25-d196-4000-818f-f3a11d173c03 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:44.630445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b9c25-d196-4000-818f-f3a11d173c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-0b743f23-639e-4713-955d-b359ed8ef9cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:44.642197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:44.643163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:44.644076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:44.644076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:44.644076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:44.644076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:44.644076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 223/459] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:44.654667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b743f23-639e-4713-955d-b359ed8ef9cc None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:44.664606 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f83b9c25-d196-4000-818f-f3a11d173c03 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:44.666104 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f83b9c25-d196-4000-818f-f3a11d173c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:44.678681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:44.754974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 237/460] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:44.754974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310e611-c93c-45b8-9961-843810faaa04 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:44.754974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310e611-c93c-45b8-9961-843810faaa04 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:44.754974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310e611-c93c-45b8-9961-843810faaa04 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:44.756136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 238/461] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:44.765650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba333fc-988c-4c74-b1f5-b6ff9d6fc2ce None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:44.765951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba333fc-988c-4c74-b1f5-b6ff9d6fc2ce None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:44.766381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba333fc-988c-4c74-b1f5-b6ff9d6fc2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:44.787032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-4ba333fc-988c-4c74-b1f5-b6ff9d6fc2ce None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:44.788331 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4ba333fc-988c-4c74-b1f5-b6ff9d6fc2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:44.808651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b743f23-639e-4713-955d-b359ed8ef9cc None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:44.846331 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b37f1-6181-45a4-842e-17c2176c7a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:44.847919 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b37f1-6181-45a4-842e-17c2176c7a7f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:44.848987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b37f1-6181-45a4-842e-17c2176c7a7f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:44.849458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b37f1-6181-45a4-842e-17c2176c7a7f None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:44.850466 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b37f1-6181-45a4-842e-17c2176c7a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:44.851031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b37f1-6181-45a4-842e-17c2176c7a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:44.852605 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b37f1-6181-45a4-842e-17c2176c7a7f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:44.866837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 224/462] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:44.867934 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b37f1-6181-45a4-842e-17c2176c7a7f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:44.886975 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 239/463] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:40:44.887854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba333fc-988c-4c74-b1f5-b6ff9d6fc2ce None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:44.939648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:44.959321 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:44.983299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b37f1-6181-45a4-842e-17c2176c7a7f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:45.012936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba333fc-988c-4c74-b1f5-b6ff9d6fc2ce None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:45.018408 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bad794-91d2-4dd4-a732-117a5de966d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:45.020102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bad794-91d2-4dd4-a732-117a5de966d4 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:45.020390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bad794-91d2-4dd4-a732-117a5de966d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:45.020662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bad794-91d2-4dd4-a732-117a5de966d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:45.031549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bad794-91d2-4dd4-a732-117a5de966d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:45.033084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bad794-91d2-4dd4-a732-117a5de966d4 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:45.048950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6105b0-1201-4086-865f-25f0edf2364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:45.049624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6105b0-1201-4086-865f-25f0edf2364b None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:45.049911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6105b0-1201-4086-865f-25f0edf2364b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:45.050289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6105b0-1201-4086-865f-25f0edf2364b None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:45.051070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6105b0-1201-4086-865f-25f0edf2364b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-f4bad794-91d2-4dd4-a732-117a5de966d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:45.051873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:45.052702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:45.053536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:45.053536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:45.053536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:45.053536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:45.053741 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6105b0-1201-4086-865f-25f0edf2364b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:45.058529 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 225/464] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:45.059276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bad794-91d2-4dd4-a732-117a5de966d4 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:45.081475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 240/465] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:40:44 2022] POST /identity/v3/users => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:40:45.081475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb5fa4-14fa-417b-8438-14960bd7f31d None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:45.081475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb5fa4-14fa-417b-8438-14960bd7f31d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:45.081475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb5fa4-14fa-417b-8438-14960bd7f31d None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:45.086906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 241/466] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:45.086906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb5fa4-14fa-417b-8438-14960bd7f31d None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:45.201581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:45.216864 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb5fa4-14fa-417b-8438-14960bd7f31d None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:45.222989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bad794-91d2-4dd4-a732-117a5de966d4 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:45.260362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089bdf7-3db6-4856-9877-a518d70088ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:45.261089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089bdf7-3db6-4856-9877-a518d70088ee None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:45.261464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089bdf7-3db6-4856-9877-a518d70088ee None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:45.261811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089bdf7-3db6-4856-9877-a518d70088ee None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:45.262758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089bdf7-3db6-4856-9877-a518d70088ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:45.263172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089bdf7-3db6-4856-9877-a518d70088ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:45.264770 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089bdf7-3db6-4856-9877-a518d70088ee None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:45.266399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:45.274098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f779d-d4ec-4523-8f89-7701b598b447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:45.275281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f779d-d4ec-4523-8f89-7701b598b447 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:45.275716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f779d-d4ec-4523-8f89-7701b598b447 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:45.276153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f779d-d4ec-4523-8f89-7701b598b447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:45.289779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f779d-d4ec-4523-8f89-7701b598b447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:45.291679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f779d-d4ec-4523-8f89-7701b598b447 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:45.291848 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 242/467] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:45.293212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e782c3-2946-4d6c-8445-2b9ea7e8cbe7 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:45.293505 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e782c3-2946-4d6c-8445-2b9ea7e8cbe7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:45.293855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e782c3-2946-4d6c-8445-2b9ea7e8cbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-2b8f779d-d4ec-4523-8f89-7701b598b447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:45.312390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:45.313445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:45.314498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:45.314498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:45.314498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:45.314498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:45.315973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-78e782c3-2946-4d6c-8445-2b9ea7e8cbe7 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:45.316945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 226/468] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:45.318126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f779d-d4ec-4523-8f89-7701b598b447 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:45.318293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-78e782c3-2946-4d6c-8445-2b9ea7e8cbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:45.400972 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 243/469] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:45.402156 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e782c3-2946-4d6c-8445-2b9ea7e8cbe7 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:45.464061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f779d-d4ec-4523-8f89-7701b598b447 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:45.500788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:45.513098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafb204-8d26-4b28-ae30-f1352708889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:45.514577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafb204-8d26-4b28-ae30-f1352708889f None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:45.515335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafb204-8d26-4b28-ae30-f1352708889f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:45.515840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafb204-8d26-4b28-ae30-f1352708889f None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:45.517411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafb204-8d26-4b28-ae30-f1352708889f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:45.519870 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafb204-8d26-4b28-ae30-f1352708889f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:45.523470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e782c3-2946-4d6c-8445-2b9ea7e8cbe7 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:45.549977 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 227/470] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:40:45 2022] POST /identity/v3/users => generated 248 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:40:45.551672 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafb204-8d26-4b28-ae30-f1352708889f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:45.599905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b2b52-5f76-45f3-8ceb-0b93cc9df5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:45.606895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b2b52-5f76-45f3-8ceb-0b93cc9df5f8 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:45.607906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:45.608644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b2b52-5f76-45f3-8ceb-0b93cc9df5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:45.609177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b2b52-5f76-45f3-8ceb-0b93cc9df5f8 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:45.610210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b2b52-5f76-45f3-8ceb-0b93cc9df5f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:45.610742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b2b52-5f76-45f3-8ceb-0b93cc9df5f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:45.612542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b2b52-5f76-45f3-8ceb-0b93cc9df5f8 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:45.635821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 244/471] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:45.644202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b2b52-5f76-45f3-8ceb-0b93cc9df5f8 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:45.682516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafb204-8d26-4b28-ae30-f1352708889f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:45.726297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be32c7-ca20-4cc0-9343-ec13bda28d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:45.727208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be32c7-ca20-4cc0-9343-ec13bda28d93 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:45.727495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be32c7-ca20-4cc0-9343-ec13bda28d93 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:45.727820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be32c7-ca20-4cc0-9343-ec13bda28d93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:45.740631 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be32c7-ca20-4cc0-9343-ec13bda28d93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:45.742233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be32c7-ca20-4cc0-9343-ec13bda28d93 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-c9be32c7-ca20-4cc0-9343-ec13bda28d93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:45.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:45.763081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:45.763933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:45.763933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:45.763933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:45.763933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:45.763933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 228/472] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:45.764756 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028aec8-4f63-48d1-8e71-1a67e948a578 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:45.765041 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028aec8-4f63-48d1-8e71-1a67e948a578 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:45.765317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028aec8-4f63-48d1-8e71-1a67e948a578 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:45.771414 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 229/473] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:45.771943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028aec8-4f63-48d1-8e71-1a67e948a578 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:45.802435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b2b52-5f76-45f3-8ceb-0b93cc9df5f8 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:45.851404 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69280922-0a82-4add-8655-b575957c445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:45.852349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69280922-0a82-4add-8655-b575957c445d None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:45.852639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69280922-0a82-4add-8655-b575957c445d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:45.852927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69280922-0a82-4add-8655-b575957c445d None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:45.853651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69280922-0a82-4add-8655-b575957c445d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:45.856137 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69280922-0a82-4add-8655-b575957c445d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:45.867896 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:45.881560 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:45.885005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 245/474] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:40:45 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:40:45.886529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0020df-8395-49e9-b647-055d008ff9a8 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:45.891173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0020df-8395-49e9-b647-055d008ff9a8 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:45.891631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0020df-8395-49e9-b647-055d008ff9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:45.903832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028aec8-4f63-48d1-8e71-1a67e948a578 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:45.929876 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-fa0020df-8395-49e9-b647-055d008ff9a8 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:45.931143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-fa0020df-8395-49e9-b647-055d008ff9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:45.975858 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549602d-a634-4a68-b700-ae9b263ee625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:45.976543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549602d-a634-4a68-b700-ae9b263ee625 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:45.976826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549602d-a634-4a68-b700-ae9b263ee625 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:45.977099 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549602d-a634-4a68-b700-ae9b263ee625 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:45.977838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549602d-a634-4a68-b700-ae9b263ee625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:45.978144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549602d-a634-4a68-b700-ae9b263ee625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:45.979597 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549602d-a634-4a68-b700-ae9b263ee625 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:46.001688 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 230/475] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:46.008230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549602d-a634-4a68-b700-ae9b263ee625 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:46.080669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 246/476] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:46.104968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0020df-8395-49e9-b647-055d008ff9a8 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:46.155247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549602d-a634-4a68-b700-ae9b263ee625 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:46.193216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06075516-4df3-4fe2-9383-c351d4c300a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:46.193890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06075516-4df3-4fe2-9383-c351d4c300a9 None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:46.194175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06075516-4df3-4fe2-9383-c351d4c300a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:46.194449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06075516-4df3-4fe2-9383-c351d4c300a9 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:46.195302 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06075516-4df3-4fe2-9383-c351d4c300a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:46.196841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06075516-4df3-4fe2-9383-c351d4c300a9 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:46.224180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 231/477] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:40:46 2022] POST /identity/v3/users => generated 249 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:40:46.256603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0020df-8395-49e9-b647-055d008ff9a8 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:46.289596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:46.300739 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9f227-98dd-40ab-8708-aad84c12f669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:46.301897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9f227-98dd-40ab-8708-aad84c12f669 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:46.302544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9f227-98dd-40ab-8708-aad84c12f669 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:46.303287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9f227-98dd-40ab-8708-aad84c12f669 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:46.304065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9f227-98dd-40ab-8708-aad84c12f669 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:46.305636 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9f227-98dd-40ab-8708-aad84c12f669 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:46.314039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.sql.core [None req-25b9f227-98dd-40ab-8708-aad84c12f669 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 13:40:46.314039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 13:40:46.314039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [parameters: {'id': '2e8eeceb4b944b0ebf97238683de6134', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 13:40:46.314039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86630) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-25b9f227-98dd-40ab-8708-aad84c12f669 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application result = self._query(query) May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application conn.query(q) May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application result.read() May 19 13:40:46.333577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return ref.to_dict() May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 13:40:46.334535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application next(self.gen) May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application yield resource May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application next(self.gen) May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.session.rollback() May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.force_reraise() May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise self.value May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application session.commit() May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 13:40:46.335510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._prepare_impl() May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.session.flush() May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._flush(objects) May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application compat.raise_( May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application flush_context.execute() May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rec.execute(self) May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 13:40:46.336383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 13:40:46.337253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application result = self._query(query) May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application conn.query(q) May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application result.read() May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 13:40:46.338086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [parameters: {'id': '2e8eeceb4b944b0ebf97238683de6134', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:46.339963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 13:40:46.341912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:46.348492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 247/478] 10.222.0.52 () {60 vars in 1131 bytes} [Thu May 19 13:40:46 2022] POST /identity/v3/roles => generated 144 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 13:40:46.351135 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06075516-4df3-4fe2-9383-c351d4c300a9 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:46.361665 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:46.465608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06075516-4df3-4fe2-9383-c351d4c300a9 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:46.500905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf59946-3072-45d4-bcb5-9de28e598806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:46.502074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf59946-3072-45d4-bcb5-9de28e598806 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:46.502580 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf59946-3072-45d4-bcb5-9de28e598806 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:46.503074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf59946-3072-45d4-bcb5-9de28e598806 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:46.514314 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf59946-3072-45d4-bcb5-9de28e598806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-3bf59946-3072-45d4-bcb5-9de28e598806 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:46.523303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:46.524215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:46.525014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:46.525014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:46.525014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:46.525014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:46.525014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:46.525014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:46.525014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:46.525014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:46.525014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:46.525014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:46.525014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:40:46.525014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:46.527764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 232/479] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:40:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:46.533586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9f227-98dd-40ab-8708-aad84c12f669 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:46.650353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9f227-98dd-40ab-8708-aad84c12f669 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:46.686836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73f377-874d-40e0-a4ce-c4678300a835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:46.687872 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f377-874d-40e0-a4ce-c4678300a835 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:46.688398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f377-874d-40e0-a4ce-c4678300a835 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:46.688869 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f377-874d-40e0-a4ce-c4678300a835 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:46.689820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73f377-874d-40e0-a4ce-c4678300a835 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:46.690331 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73f377-874d-40e0-a4ce-c4678300a835 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:46.691965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73f377-874d-40e0-a4ce-c4678300a835 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:46.702042 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 248/480] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:47.962061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43493732-6a62-47e7-9ed5-3581853fe7ca None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:47.963096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43493732-6a62-47e7-9ed5-3581853fe7ca None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:47.963605 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43493732-6a62-47e7-9ed5-3581853fe7ca None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:47.965455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 233/481] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:47.970514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8b98d-57fd-4028-ad67-443a6562fef0 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:47.971326 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8b98d-57fd-4028-ad67-443a6562fef0 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:47.971798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8b98d-57fd-4028-ad67-443a6562fef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:47.995681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-1ef8b98d-57fd-4028-ad67-443a6562fef0 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:47.997453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1ef8b98d-57fd-4028-ad67-443a6562fef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:48.057021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:48.091401 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 249/482] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:48.131320 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43493732-6a62-47e7-9ed5-3581853fe7ca None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:48.241682 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:48.301285 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43493732-6a62-47e7-9ed5-3581853fe7ca None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:48.356476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136e8da-2786-4de0-b807-b3f6668a6cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:48.357291 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136e8da-2786-4de0-b807-b3f6668a6cf4 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:48.357572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136e8da-2786-4de0-b807-b3f6668a6cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:48.357845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136e8da-2786-4de0-b807-b3f6668a6cf4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:48.382942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136e8da-2786-4de0-b807-b3f6668a6cf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:48.389577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437643a3-bc9c-4a53-a053-da0f887c6ff2 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:48.389884 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437643a3-bc9c-4a53-a053-da0f887c6ff2 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:48.390173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437643a3-bc9c-4a53-a053-da0f887c6ff2 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-0136e8da-2786-4de0-b807-b3f6668a6cf4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:48.390655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:48.391550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:48.392345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:48.392345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:48.392345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:48.392345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:48.392345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:48.392345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:48.392345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:48.392345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:48.392345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:48.392345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:48.392345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:48.392345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:48.392951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 250/483] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:48.393018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 234/484] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:40:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:48.401711 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136e8da-2786-4de0-b807-b3f6668a6cf4 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:48.402729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e847b-339a-4537-ad14-c44b2e079cd2 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:48.402729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e847b-339a-4537-ad14-c44b2e079cd2 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:48.402729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e847b-339a-4537-ad14-c44b2e079cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:48.431642 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ff5e847b-339a-4537-ad14-c44b2e079cd2 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:48.432846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ff5e847b-339a-4537-ad14-c44b2e079cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:48.579263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 251/485] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:48.601324 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136e8da-2786-4de0-b807-b3f6668a6cf4 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:48.634144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:48.653936 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461abf9-cde3-4549-942c-e3a481cb6a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:48.654617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461abf9-cde3-4549-942c-e3a481cb6a78 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:48.654935 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461abf9-cde3-4549-942c-e3a481cb6a78 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:48.655219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461abf9-cde3-4549-942c-e3a481cb6a78 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:48.655977 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461abf9-cde3-4549-942c-e3a481cb6a78 None admin] RBAC: Adding query filter params (name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:48.656283 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461abf9-cde3-4549-942c-e3a481cb6a78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:48.657773 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461abf9-cde3-4549-942c-e3a481cb6a78 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:48.670170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 235/486] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:40:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:48.672569 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e847b-339a-4537-ad14-c44b2e079cd2 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:48.685087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461abf9-cde3-4549-942c-e3a481cb6a78 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:48.800269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461abf9-cde3-4549-942c-e3a481cb6a78 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:48.814918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e847b-339a-4537-ad14-c44b2e079cd2 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:48.830958 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:48.839763 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23092383-28ce-44e9-a120-18d852c1e580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:48.840567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23092383-28ce-44e9-a120-18d852c1e580 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:48.840854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23092383-28ce-44e9-a120-18d852c1e580 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:48.841131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23092383-28ce-44e9-a120-18d852c1e580 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:48.852994 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23092383-28ce-44e9-a120-18d852c1e580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:48.855229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7235f6-d1ab-4779-883a-0e6d641b00ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:48.856017 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7235f6-d1ab-4779-883a-0e6d641b00ec None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:48.856304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7235f6-d1ab-4779-883a-0e6d641b00ec None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:48.856579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7235f6-d1ab-4779-883a-0e6d641b00ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-23092383-28ce-44e9-a120-18d852c1e580 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:48.861007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:48.861974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:48.862855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:48.862855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:48.862855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:48.862855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:48.862855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:48.862855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:48.862855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:48.862855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:48.862855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:48.862855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:48.862855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:48.862855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:48.862855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 236/487] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:40:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:48.864712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60163424-e714-4d34-98a5-9ef40316d5dd None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:48.864712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60163424-e714-4d34-98a5-9ef40316d5dd None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:48.865041 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60163424-e714-4d34-98a5-9ef40316d5dd None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:48.866771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 237/488] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:48.872176 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60163424-e714-4d34-98a5-9ef40316d5dd None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:48.872718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7235f6-d1ab-4779-883a-0e6d641b00ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:48.874222 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7235f6-d1ab-4779-883a-0e6d641b00ec None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-6b7235f6-d1ab-4779-883a-0e6d641b00ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:48.895697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:48.897547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:48.899182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:48.899182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:48.899182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:48.899182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:48.899182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 252/489] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:48.899182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1035463-e697-46c8-ade6-b1a2a7b0db7b None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:48.899182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1035463-e697-46c8-ade6-b1a2a7b0db7b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:48.899182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1035463-e697-46c8-ade6-b1a2a7b0db7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:48.918876 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-b1035463-e697-46c8-ade6-b1a2a7b0db7b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:48.920116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-b1035463-e697-46c8-ade6-b1a2a7b0db7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:48.936382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:49.000937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60163424-e714-4d34-98a5-9ef40316d5dd None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:49.027350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 253/490] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:49.028039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1035463-e697-46c8-ade6-b1a2a7b0db7b None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:49.055390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e2439-8cf4-48a3-8072-e51d5498c6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:49.056062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e2439-8cf4-48a3-8072-e51d5498c6cf None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.056347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e2439-8cf4-48a3-8072-e51d5498c6cf None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.056627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e2439-8cf4-48a3-8072-e51d5498c6cf None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.057381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e2439-8cf4-48a3-8072-e51d5498c6cf None admin] RBAC: Adding query filter params (name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:49.057685 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e2439-8cf4-48a3-8072-e51d5498c6cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:49.059067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e2439-8cf4-48a3-8072-e51d5498c6cf None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:49.070153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 238/491] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:40:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:40:49.071021 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e2439-8cf4-48a3-8072-e51d5498c6cf None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:49.169061 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:49.179589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1035463-e697-46c8-ade6-b1a2a7b0db7b None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:49.182407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e2439-8cf4-48a3-8072-e51d5498c6cf None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:49.194335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:49.214225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfe160-fa6b-4300-8d60-273c765c5dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:49.214906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfe160-fa6b-4300-8d60-273c765c5dfc None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.215197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfe160-fa6b-4300-8d60-273c765c5dfc None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.215477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfe160-fa6b-4300-8d60-273c765c5dfc None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.216210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfe160-fa6b-4300-8d60-273c765c5dfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:49.216506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfe160-fa6b-4300-8d60-273c765c5dfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:49.217863 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfe160-fa6b-4300-8d60-273c765c5dfc None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:49.221767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8227435-df73-49d4-87ad-d6118387d07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:49.223480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8227435-df73-49d4-87ad-d6118387d07e None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.223947 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8227435-df73-49d4-87ad-d6118387d07e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.224408 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8227435-df73-49d4-87ad-d6118387d07e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.233978 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 254/492] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:49.233978 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfe160-fa6b-4300-8d60-273c765c5dfc None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:49.241304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8227435-df73-49d4-87ad-d6118387d07e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-a8227435-df73-49d4-87ad-d6118387d07e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:49.250427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:49.251405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:49.252188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:49.252188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:49.252188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:49.252188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:49.252188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:49.252188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:49.252188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:49.252188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:49.252188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:49.252188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:49.252188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:49.252188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:49.252188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 239/493] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:40:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:49.253708 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb766ea-40e6-4989-b488-b020669a1980 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.253708 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb766ea-40e6-4989-b488-b020669a1980 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.253708 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb766ea-40e6-4989-b488-b020669a1980 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.256092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 240/494] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:49.260025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb766ea-40e6-4989-b488-b020669a1980 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:49.342151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfe160-fa6b-4300-8d60-273c765c5dfc None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:49.382528 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb766ea-40e6-4989-b488-b020669a1980 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:49.388241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9cc60-133e-46e7-96e4-29927fb619ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:49.389361 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9cc60-133e-46e7-96e4-29927fb619ed None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.389855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9cc60-133e-46e7-96e4-29927fb619ed None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.390141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9cc60-133e-46e7-96e4-29927fb619ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.404979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9cc60-133e-46e7-96e4-29927fb619ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:49.406453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9cc60-133e-46e7-96e4-29927fb619ed None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:49.421536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9190b3e-ce25-4b3d-805f-0fef9a192e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:49.422240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9190b3e-ce25-4b3d-805f-0fef9a192e31 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.422523 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9190b3e-ce25-4b3d-805f-0fef9a192e31 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.422798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9190b3e-ce25-4b3d-805f-0fef9a192e31 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-c3a9cc60-133e-46e7-96e4-29927fb619ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:49.425052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:49.426434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:49.427409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:49.427409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:49.427409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:49.427409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:49.427409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 255/495] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:49.428658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9cc60-133e-46e7-96e4-29927fb619ed None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:49.433878 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9190b3e-ce25-4b3d-805f-0fef9a192e31 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:49.435357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9190b3e-ce25-4b3d-805f-0fef9a192e31 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:49.437720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-b9190b3e-ce25-4b3d-805f-0fef9a192e31 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:40:49.451616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:49.452570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:49.453470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:49.453470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:49.453470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:40:49.453470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:40:49.453470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:40:49.453470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:40:49.453470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:40:49.453470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:40:49.453470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:40:49.453470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:49.453470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 241/496] 10.222.0.52 () {56 vars in 1093 bytes} [Thu May 19 13:40:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:49.455137 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb816969-9318-4a37-b3ec-72071f5abc46 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.455137 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb816969-9318-4a37-b3ec-72071f5abc46 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.455137 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb816969-9318-4a37-b3ec-72071f5abc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.471740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-bb816969-9318-4a37-b3ec-72071f5abc46 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:49.473523 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-bb816969-9318-4a37-b3ec-72071f5abc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:49.497457 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:49.544518 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9cc60-133e-46e7-96e4-29927fb619ed None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:49.545550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 242/497] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:49.546393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb816969-9318-4a37-b3ec-72071f5abc46 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:49.578853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7c744-7c3a-4761-9c71-a840b34f6b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:49.579543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7c744-7c3a-4761-9c71-a840b34f6b83 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.579828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7c744-7c3a-4761-9c71-a840b34f6b83 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.580099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7c744-7c3a-4761-9c71-a840b34f6b83 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.580837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7c744-7c3a-4761-9c71-a840b34f6b83 None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:49.581140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7c744-7c3a-4761-9c71-a840b34f6b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:49.583086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7c744-7c3a-4761-9c71-a840b34f6b83 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:49.590988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 256/498] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:40:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:49.591773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7c744-7c3a-4761-9c71-a840b34f6b83 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:49.653269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb816969-9318-4a37-b3ec-72071f5abc46 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:49.657508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:49.686631 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24173d0f-2c28-4fa0-a361-0d7325ce1a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:49.687592 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24173d0f-2c28-4fa0-a361-0d7325ce1a8f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.687705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24173d0f-2c28-4fa0-a361-0d7325ce1a8f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.687903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24173d0f-2c28-4fa0-a361-0d7325ce1a8f None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.688619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24173d0f-2c28-4fa0-a361-0d7325ce1a8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:49.688920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24173d0f-2c28-4fa0-a361-0d7325ce1a8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:49.690287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24173d0f-2c28-4fa0-a361-0d7325ce1a8f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:49.694089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7c744-7c3a-4761-9c71-a840b34f6b83 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:49.701996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 243/499] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:49.703358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18331dee-6cd5-4ab8-ba0e-cfd706812034 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.703654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18331dee-6cd5-4ab8-ba0e-cfd706812034 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.703952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18331dee-6cd5-4ab8-ba0e-cfd706812034 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.705791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 244/500] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:49.707472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18331dee-6cd5-4ab8-ba0e-cfd706812034 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:49.729603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:49.735002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce8cf4-1a83-46e1-81aa-8aaa691851a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:49.735727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce8cf4-1a83-46e1-81aa-8aaa691851a5 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.736013 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce8cf4-1a83-46e1-81aa-8aaa691851a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.736316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce8cf4-1a83-46e1-81aa-8aaa691851a5 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.737149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce8cf4-1a83-46e1-81aa-8aaa691851a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:49.737463 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce8cf4-1a83-46e1-81aa-8aaa691851a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:49.738932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce8cf4-1a83-46e1-81aa-8aaa691851a5 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:49.775155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 257/501] 10.222.0.52 () {56 vars in 1133 bytes} [Thu May 19 13:40:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:49.775974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce8cf4-1a83-46e1-81aa-8aaa691851a5 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:49.813706 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18331dee-6cd5-4ab8-ba0e-cfd706812034 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:49.847935 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c99d4-1b90-4421-9be6-9e0af719ccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:49.848728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c99d4-1b90-4421-9be6-9e0af719ccf4 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.849010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c99d4-1b90-4421-9be6-9e0af719ccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.849284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c99d4-1b90-4421-9be6-9e0af719ccf4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.859223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c99d4-1b90-4421-9be6-9e0af719ccf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-a20c99d4-1b90-4421-9be6-9e0af719ccf4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:49.866852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:49.867753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:49.868555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:49.868555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:49.868555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:49.868555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:49.868555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:49.868555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:49.868555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:49.868555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:49.868555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:49.868555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:49.868555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:49.868555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:49.868555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 245/502] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:40:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:49.869207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c99d4-1b90-4421-9be6-9e0af719ccf4 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:49.877046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce8cf4-1a83-46e1-81aa-8aaa691851a5 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:49.905454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:49.914906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d044f7-c883-4818-af36-9aa16aca3697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:49.915700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d044f7-c883-4818-af36-9aa16aca3697 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.915983 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d044f7-c883-4818-af36-9aa16aca3697 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.916266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d044f7-c883-4818-af36-9aa16aca3697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.926117 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d044f7-c883-4818-af36-9aa16aca3697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:49.927596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d044f7-c883-4818-af36-9aa16aca3697 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-90d044f7-c883-4818-af36-9aa16aca3697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:49.945742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:49.946765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:49.947640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:49.947640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:49.947640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:49.947640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:49.950699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:49.951015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 258/503] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:49.952342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7358f-e0b2-4007-b422-b7ed42af6b76 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:49.952679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7358f-e0b2-4007-b422-b7ed42af6b76 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:49.953049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7358f-e0b2-4007-b422-b7ed42af6b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:49.970039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-acd7358f-e0b2-4007-b422-b7ed42af6b76 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:49.970955 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c99d4-1b90-4421-9be6-9e0af719ccf4 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:49.971271 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-acd7358f-e0b2-4007-b422-b7ed42af6b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:50.005286 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cc75a-2c42-4736-bd90-3afdf36f26fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:50.006071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cc75a-2c42-4736-bd90-3afdf36f26fb None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:50.006355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cc75a-2c42-4736-bd90-3afdf36f26fb None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:50.006629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cc75a-2c42-4736-bd90-3afdf36f26fb None admin] PATH_INFO: `/v3/users/glance` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:50.014132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7cc75a-2c42-4736-bd90-3afdf36f26fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:50.015596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7cc75a-2c42-4736-bd90-3afdf36f26fb None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-fe7cc75a-2c42-4736-bd90-3afdf36f26fb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:40:50.030053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:50.030992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:50.031896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:50.031896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:50.031896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:40:50.031896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:40:50.031896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:40:50.031896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:40:50.031896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:40:50.031896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:40:50.031896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 13:40:50.031896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:50.031896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 246/504] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:50.033041 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cc75a-2c42-4736-bd90-3afdf36f26fb None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:50.041004 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 259/505] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:50.041840 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7358f-e0b2-4007-b422-b7ed42af6b76 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:50.140460 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cc75a-2c42-4736-bd90-3afdf36f26fb None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:50.140852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7358f-e0b2-4007-b422-b7ed42af6b76 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:50.150548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:50.173523 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad450a90-7d8e-4399-935d-81084c7df52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:50.174162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad450a90-7d8e-4399-935d-81084c7df52a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:50.174443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad450a90-7d8e-4399-935d-81084c7df52a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:50.174716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad450a90-7d8e-4399-935d-81084c7df52a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:50.176804 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13158d6-9a38-4922-aae0-f4a072351e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:50.177470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13158d6-9a38-4922-aae0-f4a072351e12 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:50.177837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13158d6-9a38-4922-aae0-f4a072351e12 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:50.178032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13158d6-9a38-4922-aae0-f4a072351e12 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:50.178787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13158d6-9a38-4922-aae0-f4a072351e12 None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:50.179120 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13158d6-9a38-4922-aae0-f4a072351e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:50.180506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13158d6-9a38-4922-aae0-f4a072351e12 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:50.185601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:50.188531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 260/506] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:40:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:50.189360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13158d6-9a38-4922-aae0-f4a072351e12 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:50.247071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad450a90-7d8e-4399-935d-81084c7df52a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:50.248566 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad450a90-7d8e-4399-935d-81084c7df52a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:50.291038 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13158d6-9a38-4922-aae0-f4a072351e12 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:50.323823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7b0ff-8506-4c53-8092-cb92fb02ebb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:50.324480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7b0ff-8506-4c53-8092-cb92fb02ebb1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:50.324767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7b0ff-8506-4c53-8092-cb92fb02ebb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:50.325051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7b0ff-8506-4c53-8092-cb92fb02ebb1 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:50.325769 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7b0ff-8506-4c53-8092-cb92fb02ebb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:50.326077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7b0ff-8506-4c53-8092-cb92fb02ebb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:50.327497 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7b0ff-8506-4c53-8092-cb92fb02ebb1 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:50.339227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 261/507] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:50.340025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7b0ff-8506-4c53-8092-cb92fb02ebb1 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:50.348501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 247/508] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:50.349347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad450a90-7d8e-4399-935d-81084c7df52a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:50.371909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:50.393802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:50.441472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7b0ff-8506-4c53-8092-cb92fb02ebb1 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:50.452114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad450a90-7d8e-4399-935d-81084c7df52a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:50.475231 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c6894-2ebc-4940-835d-aafb2b302ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:50.475900 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c6894-2ebc-4940-835d-aafb2b302ed0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:50.476185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c6894-2ebc-4940-835d-aafb2b302ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:50.476462 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c6894-2ebc-4940-835d-aafb2b302ed0 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:50.477292 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c6894-2ebc-4940-835d-aafb2b302ed0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:50.477605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c6894-2ebc-4940-835d-aafb2b302ed0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:50.479067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c6894-2ebc-4940-835d-aafb2b302ed0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:50.486369 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20d6cd-34e0-409d-a93c-9f8e7f3bb1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:50.487291 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20d6cd-34e0-409d-a93c-9f8e7f3bb1e2 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:50.487579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20d6cd-34e0-409d-a93c-9f8e7f3bb1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:50.488375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20d6cd-34e0-409d-a93c-9f8e7f3bb1e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:50.496279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 262/509] 10.222.0.52 () {56 vars in 1137 bytes} [Thu May 19 13:40:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:50.497394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c6894-2ebc-4940-835d-aafb2b302ed0 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:50.502673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20d6cd-34e0-409d-a93c-9f8e7f3bb1e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:50.504825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20d6cd-34e0-409d-a93c-9f8e7f3bb1e2 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-0e20d6cd-34e0-409d-a93c-9f8e7f3bb1e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:50.522382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:50.523309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:50.524145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:50.524145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:50.524145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:50.524145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:50.524145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 248/510] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:50.524621 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20d6cd-34e0-409d-a93c-9f8e7f3bb1e2 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:50.585669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:50.598790 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c6894-2ebc-4940-835d-aafb2b302ed0 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:50.617458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:50.626501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20d6cd-34e0-409d-a93c-9f8e7f3bb1e2 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:50.633538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da7550-55a7-4564-9c1e-65a0da911efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:50.634322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da7550-55a7-4564-9c1e-65a0da911efd None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:50.634598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da7550-55a7-4564-9c1e-65a0da911efd None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:50.634906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da7550-55a7-4564-9c1e-65a0da911efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:50.645369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8da7550-55a7-4564-9c1e-65a0da911efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:50.646808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8da7550-55a7-4564-9c1e-65a0da911efd None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:50.661782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bb577-48f7-4902-8601-8e02f1901c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:50.662566 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bb577-48f7-4902-8601-8e02f1901c28 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:50.662853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bb577-48f7-4902-8601-8e02f1901c28 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:50.663163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bb577-48f7-4902-8601-8e02f1901c28 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-b8da7550-55a7-4564-9c1e-65a0da911efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:50.663895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:50.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:50.665686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:50.665686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:50.665686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:50.665686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:50.665686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 263/511] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:50.666151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da7550-55a7-4564-9c1e-65a0da911efd None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:50.674228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bb577-48f7-4902-8601-8e02f1901c28 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:50.675728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bb577-48f7-4902-8601-8e02f1901c28 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-330bb577-48f7-4902-8601-8e02f1901c28 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:40:50.689979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:50.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:50.692604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:50.692604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:50.692604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:40:50.692604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:40:50.692604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:40:50.692604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:40:50.692604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:40:50.692604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:40:50.692604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:40:50.692604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:50.692604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 249/512] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:50.692604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bb577-48f7-4902-8601-8e02f1901c28 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:50.768298 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da7550-55a7-4564-9c1e-65a0da911efd None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:50.796213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bb577-48f7-4902-8601-8e02f1901c28 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:50.801811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73174e39-06eb-49c3-abab-8c83e304ffed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:50.802602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73174e39-06eb-49c3-abab-8c83e304ffed None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:50.802913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73174e39-06eb-49c3-abab-8c83e304ffed None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:50.803197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73174e39-06eb-49c3-abab-8c83e304ffed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:50.811850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:50.814139 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73174e39-06eb-49c3-abab-8c83e304ffed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:50.816611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73174e39-06eb-49c3-abab-8c83e304ffed None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:50.830604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e732dce-4835-4bcf-aa4b-e0741cf8e515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:50.831277 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e732dce-4835-4bcf-aa4b-e0741cf8e515 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:50.831608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e732dce-4835-4bcf-aa4b-e0741cf8e515 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:50.832438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e732dce-4835-4bcf-aa4b-e0741cf8e515 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-73174e39-06eb-49c3-abab-8c83e304ffed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:50.834584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:50.835452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:50.836689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:50.836689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:50.836689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:50.836689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:50.836689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 264/513] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:50.837192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73174e39-06eb-49c3-abab-8c83e304ffed None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:50.854656 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:50.901712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e732dce-4835-4bcf-aa4b-e0741cf8e515 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:50.903265 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e732dce-4835-4bcf-aa4b-e0741cf8e515 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:50.934582 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73174e39-06eb-49c3-abab-8c83e304ffed None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:50.967213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45233a5-e79b-4613-89d6-5073c896ae48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:50.967861 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45233a5-e79b-4613-89d6-5073c896ae48 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:50.968140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45233a5-e79b-4613-89d6-5073c896ae48 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:50.968504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45233a5-e79b-4613-89d6-5073c896ae48 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:50.969218 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45233a5-e79b-4613-89d6-5073c896ae48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:50.969521 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45233a5-e79b-4613-89d6-5073c896ae48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:50.970905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45233a5-e79b-4613-89d6-5073c896ae48 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:50.982606 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 265/514] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:50.983448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45233a5-e79b-4613-89d6-5073c896ae48 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:51.001089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 250/515] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:51.002117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e732dce-4835-4bcf-aa4b-e0741cf8e515 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:51.037557 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:51.053634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:51.085119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45233a5-e79b-4613-89d6-5073c896ae48 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:51.100779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e732dce-4835-4bcf-aa4b-e0741cf8e515 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:51.119599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398352b-eb4e-4ed6-a08a-989a1929e857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:51.120259 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398352b-eb4e-4ed6-a08a-989a1929e857 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:51.120543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398352b-eb4e-4ed6-a08a-989a1929e857 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:51.120821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398352b-eb4e-4ed6-a08a-989a1929e857 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:51.121800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398352b-eb4e-4ed6-a08a-989a1929e857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:51.122107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398352b-eb4e-4ed6-a08a-989a1929e857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:51.123588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398352b-eb4e-4ed6-a08a-989a1929e857 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:51.135197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 266/516] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:51.135983 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398352b-eb4e-4ed6-a08a-989a1929e857 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:51.136152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b994b-926d-4250-8fd9-508bb65364a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:51.136784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b994b-926d-4250-8fd9-508bb65364a7 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:51.137072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b994b-926d-4250-8fd9-508bb65364a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:51.137350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b994b-926d-4250-8fd9-508bb65364a7 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:51.138185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b994b-926d-4250-8fd9-508bb65364a7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:51.138498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b994b-926d-4250-8fd9-508bb65364a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:51.140223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b994b-926d-4250-8fd9-508bb65364a7 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:51.176667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 251/517] 10.222.0.52 () {56 vars in 1137 bytes} [Thu May 19 13:40:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:51.177473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b994b-926d-4250-8fd9-508bb65364a7 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:51.238734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398352b-eb4e-4ed6-a08a-989a1929e857 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:51.251237 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:51.273720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848ba34-9ff3-437c-b191-63f45783c789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:51.274380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848ba34-9ff3-437c-b191-63f45783c789 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:51.274668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848ba34-9ff3-437c-b191-63f45783c789 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:51.274986 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848ba34-9ff3-437c-b191-63f45783c789 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:51.275077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b994b-926d-4250-8fd9-508bb65364a7 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:51.275711 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848ba34-9ff3-437c-b191-63f45783c789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:51.276023 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848ba34-9ff3-437c-b191-63f45783c789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:51.277402 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848ba34-9ff3-437c-b191-63f45783c789 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:51.285559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:51.289278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 267/518] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:51.290013 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848ba34-9ff3-437c-b191-63f45783c789 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:51.313536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472b2dc-4350-4f72-aa89-3a6e180d1a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:51.314193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472b2dc-4350-4f72-aa89-3a6e180d1a36 None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:51.314474 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472b2dc-4350-4f72-aa89-3a6e180d1a36 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:51.314747 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472b2dc-4350-4f72-aa89-3a6e180d1a36 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:51.315468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472b2dc-4350-4f72-aa89-3a6e180d1a36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:51.317050 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472b2dc-4350-4f72-aa89-3a6e180d1a36 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:51.336091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 252/519] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:40:51 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:40:51.336884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472b2dc-4350-4f72-aa89-3a6e180d1a36 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:51.398790 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848ba34-9ff3-437c-b191-63f45783c789 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:51.437498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea171ca6-75b7-4d5b-8c56-55e6cdf6f94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:51.438326 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea171ca6-75b7-4d5b-8c56-55e6cdf6f94c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:51.438614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea171ca6-75b7-4d5b-8c56-55e6cdf6f94c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:51.438910 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea171ca6-75b7-4d5b-8c56-55e6cdf6f94c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:51.450719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea171ca6-75b7-4d5b-8c56-55e6cdf6f94c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:51.452467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea171ca6-75b7-4d5b-8c56-55e6cdf6f94c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:51.472990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472b2dc-4350-4f72-aa89-3a6e180d1a36 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-ea171ca6-75b7-4d5b-8c56-55e6cdf6f94c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:51.476998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:51.477892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:51.483954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:51.483954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:51.483954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:51.483954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:51.483954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 268/520] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:51.483954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea171ca6-75b7-4d5b-8c56-55e6cdf6f94c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:51.511034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffc7ba-fd26-43bc-91fe-7a2658ac052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:51.511945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffc7ba-fd26-43bc-91fe-7a2658ac052a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:51.512240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffc7ba-fd26-43bc-91fe-7a2658ac052a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:51.512517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffc7ba-fd26-43bc-91fe-7a2658ac052a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:51.512714 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:51.521964 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffc7ba-fd26-43bc-91fe-7a2658ac052a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:51.523471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffc7ba-fd26-43bc-91fe-7a2658ac052a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-0dffc7ba-fd26-43bc-91fe-7a2658ac052a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:40:51.539792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:51.540673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:51.541558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:51.541558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:51.541558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:40:51.541558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:40:51.541558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:40:51.541558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:40:51.541558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:40:51.541558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:40:51.541558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 13:40:51.541558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:51.541558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 253/521] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:40:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:51.542113 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffc7ba-fd26-43bc-91fe-7a2658ac052a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:51.574134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:51.607268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea171ca6-75b7-4d5b-8c56-55e6cdf6f94c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:51.644778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1995d1-295a-4262-b5fb-aa2d7e7a5558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:51.645485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1995d1-295a-4262-b5fb-aa2d7e7a5558 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:51.645779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1995d1-295a-4262-b5fb-aa2d7e7a5558 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:51.646024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1995d1-295a-4262-b5fb-aa2d7e7a5558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:51.656669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffc7ba-fd26-43bc-91fe-7a2658ac052a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:51.691827 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f629e43-99fa-4b86-ae36-a4877ee305d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:51.692773 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f629e43-99fa-4b86-ae36-a4877ee305d1 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:51.693061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f629e43-99fa-4b86-ae36-a4877ee305d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:51.693335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f629e43-99fa-4b86-ae36-a4877ee305d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:51.700612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f629e43-99fa-4b86-ae36-a4877ee305d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:51.702000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f629e43-99fa-4b86-ae36-a4877ee305d1 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-3f629e43-99fa-4b86-ae36-a4877ee305d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:40:51.717284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:51.718189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:51.719071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:40:51.719071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:40:51.719071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:40:51.719071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:40:51.719071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:40:51.719071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:40:51.719071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:51.719071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:51.723267 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 254/522] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:51.723984 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f629e43-99fa-4b86-ae36-a4877ee305d1 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:51.725675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1995d1-295a-4262-b5fb-aa2d7e7a5558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:51.727469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1995d1-295a-4262-b5fb-aa2d7e7a5558 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:51.754387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:51.819177 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:51.837917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f629e43-99fa-4b86-ae36-a4877ee305d1 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:51.838398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 269/523] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:51.839368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1995d1-295a-4262-b5fb-aa2d7e7a5558 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:51.879844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055265e9-0cb3-4185-92d0-a7bb3a9390ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:51.880640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055265e9-0cb3-4185-92d0-a7bb3a9390ef None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:51.880925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055265e9-0cb3-4185-92d0-a7bb3a9390ef None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:51.881200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055265e9-0cb3-4185-92d0-a7bb3a9390ef None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:51.881917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055265e9-0cb3-4185-92d0-a7bb3a9390ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:51.882225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055265e9-0cb3-4185-92d0-a7bb3a9390ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:51.884195 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055265e9-0cb3-4185-92d0-a7bb3a9390ef None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:51.900749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 255/524] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:51.901394 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055265e9-0cb3-4185-92d0-a7bb3a9390ef None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:51.945657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1995d1-295a-4262-b5fb-aa2d7e7a5558 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:51.968638 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:51.982224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b7e1c-af28-47a3-9ff2-0ceb81e43090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:51.982920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b7e1c-af28-47a3-9ff2-0ceb81e43090 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:51.983211 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b7e1c-af28-47a3-9ff2-0ceb81e43090 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:51.983486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b7e1c-af28-47a3-9ff2-0ceb81e43090 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:51.984317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b7e1c-af28-47a3-9ff2-0ceb81e43090 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:51.984627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b7e1c-af28-47a3-9ff2-0ceb81e43090 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:51.986054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b7e1c-af28-47a3-9ff2-0ceb81e43090 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:52.009127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 270/525] 10.222.0.52 () {56 vars in 1139 bytes} [Thu May 19 13:40:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:52.009127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b7e1c-af28-47a3-9ff2-0ceb81e43090 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:52.009373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055265e9-0cb3-4185-92d0-a7bb3a9390ef None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:52.044090 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2839ca8-15d4-43e5-a32a-bb7631412a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:52.044746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2839ca8-15d4-43e5-a32a-bb7631412a68 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:52.045028 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2839ca8-15d4-43e5-a32a-bb7631412a68 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:52.045300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2839ca8-15d4-43e5-a32a-bb7631412a68 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:52.046076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2839ca8-15d4-43e5-a32a-bb7631412a68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:52.046380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2839ca8-15d4-43e5-a32a-bb7631412a68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:52.047827 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2839ca8-15d4-43e5-a32a-bb7631412a68 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:52.071767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 256/526] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:40:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:52.071767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2839ca8-15d4-43e5-a32a-bb7631412a68 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:52.077183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:52.117805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b7e1c-af28-47a3-9ff2-0ceb81e43090 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:52.153489 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88534504-25eb-4d91-a7b9-ed0f064a14e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:52.154287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88534504-25eb-4d91-a7b9-ed0f064a14e5 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:52.154583 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88534504-25eb-4d91-a7b9-ed0f064a14e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:52.154848 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88534504-25eb-4d91-a7b9-ed0f064a14e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:52.165271 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88534504-25eb-4d91-a7b9-ed0f064a14e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:52.166738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88534504-25eb-4d91-a7b9-ed0f064a14e5 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:52.176506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2839ca8-15d4-43e5-a32a-bb7631412a68 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-88534504-25eb-4d91-a7b9-ed0f064a14e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:52.184850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:52.185970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:52.186998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:52.186998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:52.186998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:52.186998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:52.186998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 271/527] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:52.187330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88534504-25eb-4d91-a7b9-ed0f064a14e5 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:52.217913 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba721b47-c378-426f-a35e-06b6f7f291fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:52.218716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba721b47-c378-426f-a35e-06b6f7f291fa None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:52.219032 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba721b47-c378-426f-a35e-06b6f7f291fa None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:52.219338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba721b47-c378-426f-a35e-06b6f7f291fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:52.224823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:52.227636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba721b47-c378-426f-a35e-06b6f7f291fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:52.229136 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba721b47-c378-426f-a35e-06b6f7f291fa None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-ba721b47-c378-426f-a35e-06b6f7f291fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:40:52.243627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:52.244499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:52.245337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:40:52.245337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:40:52.245337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:40:52.245337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:40:52.245337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:40:52.245337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:40:52.245337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:52.245337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:52.245337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 257/528] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:52.247071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba721b47-c378-426f-a35e-06b6f7f291fa None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:52.299748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88534504-25eb-4d91-a7b9-ed0f064a14e5 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:52.324684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:52.335621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbdae0-9c71-4c9a-917c-cdab2d1355af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:52.336185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbdae0-9c71-4c9a-917c-cdab2d1355af None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:52.336474 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbdae0-9c71-4c9a-917c-cdab2d1355af None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:52.336749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbdae0-9c71-4c9a-917c-cdab2d1355af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:52.361830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba721b47-c378-426f-a35e-06b6f7f291fa None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:52.397915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd545aed-8c78-4158-a4a8-7475d69943b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:52.398570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd545aed-8c78-4158-a4a8-7475d69943b9 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:52.398849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd545aed-8c78-4158-a4a8-7475d69943b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:52.399149 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd545aed-8c78-4158-a4a8-7475d69943b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:52.399970 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd545aed-8c78-4158-a4a8-7475d69943b9 None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=4f6093d0c1fc41d98c58bace81e3d8e5) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:52.400282 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd545aed-8c78-4158-a4a8-7475d69943b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:52.401720 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd545aed-8c78-4158-a4a8-7475d69943b9 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:52.418905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 258/529] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:52 2022] GET /identity/v3/role_assignments?user.id=4f6093d0c1fc41d98c58bace81e3d8e5&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:52.418905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd545aed-8c78-4158-a4a8-7475d69943b9 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:52.430056 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbbdae0-9c71-4c9a-917c-cdab2d1355af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:52.431708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbbdae0-9c71-4c9a-917c-cdab2d1355af None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:52.469021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:52.549472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd545aed-8c78-4158-a4a8-7475d69943b9 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:52.555012 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 272/530] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:52.555950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbdae0-9c71-4c9a-917c-cdab2d1355af None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:52.587984 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:52.596207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71442e6c-50d3-4ec6-92f0-5c810a135f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:52.598108 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442e6c-50d3-4ec6-92f0-5c810a135f21 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:52.598614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442e6c-50d3-4ec6-92f0-5c810a135f21 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:52.599232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442e6c-50d3-4ec6-92f0-5c810a135f21 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:52.600078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71442e6c-50d3-4ec6-92f0-5c810a135f21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:52.600459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71442e6c-50d3-4ec6-92f0-5c810a135f21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:52.603288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71442e6c-50d3-4ec6-92f0-5c810a135f21 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:52.619925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 259/531] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:52.620911 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71442e6c-50d3-4ec6-92f0-5c810a135f21 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:52.681709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbdae0-9c71-4c9a-917c-cdab2d1355af None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:52.718382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe97a6-a77e-4a4d-9d2e-f868070ef417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:52.719080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe97a6-a77e-4a4d-9d2e-f868070ef417 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:52.719365 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe97a6-a77e-4a4d-9d2e-f868070ef417 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:52.719642 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe97a6-a77e-4a4d-9d2e-f868070ef417 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:52.720438 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe97a6-a77e-4a4d-9d2e-f868070ef417 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:52.720753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe97a6-a77e-4a4d-9d2e-f868070ef417 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:52.722248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe97a6-a77e-4a4d-9d2e-f868070ef417 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:52.724055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:52.742949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 273/532] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:40:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:52.743866 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe97a6-a77e-4a4d-9d2e-f868070ef417 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:52.747187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71442e6c-50d3-4ec6-92f0-5c810a135f21 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:52.786980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232fa56-21d1-49dd-a5b1-f296ce95e2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:52.787803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232fa56-21d1-49dd-a5b1-f296ce95e2a8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:52.788087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232fa56-21d1-49dd-a5b1-f296ce95e2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:52.788364 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232fa56-21d1-49dd-a5b1-f296ce95e2a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:52.799077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232fa56-21d1-49dd-a5b1-f296ce95e2a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:52.801053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232fa56-21d1-49dd-a5b1-f296ce95e2a8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-a232fa56-21d1-49dd-a5b1-f296ce95e2a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:52.819678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:52.820576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:52.821411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:52.821411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:52.821411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:52.821411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:52.825379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 260/533] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:52.826256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232fa56-21d1-49dd-a5b1-f296ce95e2a8 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:52.860081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe97a6-a77e-4a4d-9d2e-f868070ef417 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:52.871034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:52.900317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9e3b1-43b5-4080-bca6-381e6f30b148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:52.901561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9e3b1-43b5-4080-bca6-381e6f30b148 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:52.902101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9e3b1-43b5-4080-bca6-381e6f30b148 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:52.902623 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9e3b1-43b5-4080-bca6-381e6f30b148 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:52.910662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9e3b1-43b5-4080-bca6-381e6f30b148 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:52.912471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9e3b1-43b5-4080-bca6-381e6f30b148 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-13d9e3b1-43b5-4080-bca6-381e6f30b148 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:40:52.929033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:52.929895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:52.930743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:40:52.930743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:40:52.930743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:40:52.930743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:40:52.930743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:40:52.930743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:40:52.930743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:52.930743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:52.933053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232fa56-21d1-49dd-a5b1-f296ce95e2a8 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:52.933453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 274/534] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:52.934385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9e3b1-43b5-4080-bca6-381e6f30b148 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:52.968959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e182211-a800-4db0-bf7a-44f7a5e1f00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:52.969572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e182211-a800-4db0-bf7a-44f7a5e1f00b None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:52.969982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e182211-a800-4db0-bf7a-44f7a5e1f00b None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:52.970347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e182211-a800-4db0-bf7a-44f7a5e1f00b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:52.971302 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e182211-a800-4db0-bf7a-44f7a5e1f00b None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=666ade4ce1fe447487998d086779389b) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:52.971716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e182211-a800-4db0-bf7a-44f7a5e1f00b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:52.973845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e182211-a800-4db0-bf7a-44f7a5e1f00b None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:52.980115 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 261/535] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:52 2022] GET /identity/v3/role_assignments?user.id=666ade4ce1fe447487998d086779389b&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:52.981142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e182211-a800-4db0-bf7a-44f7a5e1f00b None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:53.000862 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:53.056139 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9e3b1-43b5-4080-bca6-381e6f30b148 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:53.092257 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e182211-a800-4db0-bf7a-44f7a5e1f00b None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:53.099231 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8544749-e82d-44bf-92bc-e665e8b89182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:53.100309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8544749-e82d-44bf-92bc-e665e8b89182 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:53.100805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8544749-e82d-44bf-92bc-e665e8b89182 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:53.101343 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8544749-e82d-44bf-92bc-e665e8b89182 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:53.101805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:53.103349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8544749-e82d-44bf-92bc-e665e8b89182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:53.103937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8544749-e82d-44bf-92bc-e665e8b89182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:53.105637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8544749-e82d-44bf-92bc-e665e8b89182 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:53.120813 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 275/536] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:53.132776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8544749-e82d-44bf-92bc-e665e8b89182 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:53.138438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1154901-1403-46ec-8c10-c2ce195db4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:53.139154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1154901-1403-46ec-8c10-c2ce195db4bb None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:53.139445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1154901-1403-46ec-8c10-c2ce195db4bb None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:53.139715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1154901-1403-46ec-8c10-c2ce195db4bb None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:53.140518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1154901-1403-46ec-8c10-c2ce195db4bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:53.140827 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1154901-1403-46ec-8c10-c2ce195db4bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:53.142260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1154901-1403-46ec-8c10-c2ce195db4bb None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:53.155652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 262/537] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:40:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:53.167993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1154901-1403-46ec-8c10-c2ce195db4bb None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:53.302906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8544749-e82d-44bf-92bc-e665e8b89182 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:53.318732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:53.334001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1154901-1403-46ec-8c10-c2ce195db4bb None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:53.351058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb047d2-ca3a-4884-88a0-7a1c700cfb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:53.351911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb047d2-ca3a-4884-88a0-7a1c700cfb59 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:53.352252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb047d2-ca3a-4884-88a0-7a1c700cfb59 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:53.352474 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb047d2-ca3a-4884-88a0-7a1c700cfb59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:53.366626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb047d2-ca3a-4884-88a0-7a1c700cfb59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:53.368115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb047d2-ca3a-4884-88a0-7a1c700cfb59 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:53.373949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1afc4-442f-4aa2-9b42-7e84f65ad592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:53.374886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1afc4-442f-4aa2-9b42-7e84f65ad592 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:53.375464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1afc4-442f-4aa2-9b42-7e84f65ad592 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:53.375955 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1afc4-442f-4aa2-9b42-7e84f65ad592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:53.377098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1afc4-442f-4aa2-9b42-7e84f65ad592 None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=1ed0973226a2446ebf7c0cc3ee04e446) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:53.377661 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1afc4-442f-4aa2-9b42-7e84f65ad592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:53.379389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1afc4-442f-4aa2-9b42-7e84f65ad592 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:53.385814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 263/538] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:53 2022] GET /identity/v3/role_assignments?user.id=1ed0973226a2446ebf7c0cc3ee04e446&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-4fb047d2-ca3a-4884-88a0-7a1c700cfb59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:40:53.391878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:53.392788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:53.393621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:40:53.393621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:40:53.393621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:40:53.393621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:40:53.393621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:40:53.393621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:40:53.393621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:53.393621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:53.407520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 276/539] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:53.409020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1afc4-442f-4aa2-9b42-7e84f65ad592 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:53.487075 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:53.575735 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1afc4-442f-4aa2-9b42-7e84f65ad592 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:53.611519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4958c05-97e8-4728-aca1-37ac857bd223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:53.612253 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4958c05-97e8-4728-aca1-37ac857bd223 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:53.612538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4958c05-97e8-4728-aca1-37ac857bd223 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:53.613428 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4958c05-97e8-4728-aca1-37ac857bd223 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:53.614228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4958c05-97e8-4728-aca1-37ac857bd223 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:53.614586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4958c05-97e8-4728-aca1-37ac857bd223 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:53.616068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4958c05-97e8-4728-aca1-37ac857bd223 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:53.633067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 264/540] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:40:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:53.637939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb047d2-ca3a-4884-88a0-7a1c700cfb59 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:53.767241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb047d2-ca3a-4884-88a0-7a1c700cfb59 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:53.803052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8dbc9-8c77-41b6-9a06-ed113e740591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:53.804071 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8dbc9-8c77-41b6-9a06-ed113e740591 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:53.804552 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8dbc9-8c77-41b6-9a06-ed113e740591 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:53.805103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8dbc9-8c77-41b6-9a06-ed113e740591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:53.806129 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8dbc9-8c77-41b6-9a06-ed113e740591 None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=f18f635bff0142409e90da66547927f1) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:53.806657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8dbc9-8c77-41b6-9a06-ed113e740591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:53.808452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8dbc9-8c77-41b6-9a06-ed113e740591 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:53.814914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 277/541] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:53 2022] GET /identity/v3/role_assignments?user.id=f18f635bff0142409e90da66547927f1&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:53.936758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1865135-bd5b-4a76-ae1a-7ba677c86532 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:53.937065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1865135-bd5b-4a76-ae1a-7ba677c86532 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:53.937347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1865135-bd5b-4a76-ae1a-7ba677c86532 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:53.939183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 265/542] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:53.946258 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78771ee-1ca7-4721-80bb-05549bf4a02e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:53.946901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78771ee-1ca7-4721-80bb-05549bf4a02e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:53.947408 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78771ee-1ca7-4721-80bb-05549bf4a02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:53.969706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-d78771ee-1ca7-4721-80bb-05549bf4a02e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:53.971308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d78771ee-1ca7-4721-80bb-05549bf4a02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:54.010411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:54.053119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 278/543] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:54.083343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1865135-bd5b-4a76-ae1a-7ba677c86532 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:54.212472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:54.227260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1865135-bd5b-4a76-ae1a-7ba677c86532 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:54.273347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff82f8-d663-4762-84c7-e414e2fa5f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:54.274455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff82f8-d663-4762-84c7-e414e2fa5f94 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:54.274982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff82f8-d663-4762-84c7-e414e2fa5f94 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:54.275440 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff82f8-d663-4762-84c7-e414e2fa5f94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:54.291333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff82f8-d663-4762-84c7-e414e2fa5f94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-c0ff82f8-d663-4762-84c7-e414e2fa5f94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:54.299363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:54.300284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:54.301273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:54.301273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:54.301273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:54.301273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:54.301273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:54.301273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:54.301273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:54.301273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:54.301273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:54.301273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:54.301273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:54.301273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:54.306956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 266/544] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:40:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:54.309186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78771ee-1ca7-4721-80bb-05549bf4a02e None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:54.456329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78771ee-1ca7-4721-80bb-05549bf4a02e None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:54.498696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba6ba8-aef1-4a7e-be11-1946efbe6ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:54.499314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba6ba8-aef1-4a7e-be11-1946efbe6ce1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:54.499595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba6ba8-aef1-4a7e-be11-1946efbe6ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:54.499867 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba6ba8-aef1-4a7e-be11-1946efbe6ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:54.500615 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba6ba8-aef1-4a7e-be11-1946efbe6ce1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:54.500917 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba6ba8-aef1-4a7e-be11-1946efbe6ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:54.502278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba6ba8-aef1-4a7e-be11-1946efbe6ce1 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:54.517485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff82f8-d663-4762-84c7-e414e2fa5f94 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:54.519437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 279/545] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:40:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:54.653111 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff82f8-d663-4762-84c7-e414e2fa5f94 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:54.692941 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1722f6-dc7c-40e2-b47e-6e69a49e631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:54.694132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1722f6-dc7c-40e2-b47e-6e69a49e631b None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:54.694619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1722f6-dc7c-40e2-b47e-6e69a49e631b None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:54.695119 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1722f6-dc7c-40e2-b47e-6e69a49e631b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:54.711961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1722f6-dc7c-40e2-b47e-6e69a49e631b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:54.713815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1722f6-dc7c-40e2-b47e-6e69a49e631b None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-1e1722f6-dc7c-40e2-b47e-6e69a49e631b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:54.737743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:54.738679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:54.739562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:54.739562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:54.739562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:54.739562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:54.747015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 267/546] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:54.750854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba6ba8-aef1-4a7e-be11-1946efbe6ce1 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:54.800491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:54.887357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba6ba8-aef1-4a7e-be11-1946efbe6ce1 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:54.934348 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321219ce-fc6c-422d-aa2e-e6d90bcc7348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:54.935046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321219ce-fc6c-422d-aa2e-e6d90bcc7348 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:54.935334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321219ce-fc6c-422d-aa2e-e6d90bcc7348 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:54.935635 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321219ce-fc6c-422d-aa2e-e6d90bcc7348 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:54.936419 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321219ce-fc6c-422d-aa2e-e6d90bcc7348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:54.936725 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321219ce-fc6c-422d-aa2e-e6d90bcc7348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:54.938108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321219ce-fc6c-422d-aa2e-e6d90bcc7348 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:54.956080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 280/547] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:54.961361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1722f6-dc7c-40e2-b47e-6e69a49e631b None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:54.969410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:55.171111 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1722f6-dc7c-40e2-b47e-6e69a49e631b None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:55.220951 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56401676-c102-43d0-a864-769ed17b00df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:55.221752 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56401676-c102-43d0-a864-769ed17b00df None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:55.222548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56401676-c102-43d0-a864-769ed17b00df None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:55.222842 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56401676-c102-43d0-a864-769ed17b00df None admin] PATH_INFO: `/v3/users/swift` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:55.236039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56401676-c102-43d0-a864-769ed17b00df None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:55.237612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56401676-c102-43d0-a864-769ed17b00df None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-56401676-c102-43d0-a864-769ed17b00df None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:40:55.256271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:55.257699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:55.259505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:55.259505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:55.259505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:40:55.259505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:40:55.259505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:40:55.259505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:40:55.259505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:40:55.259505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:40:55.259505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 13:40:55.259505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:55.262940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 268/548] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:40:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:55.264052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321219ce-fc6c-422d-aa2e-e6d90bcc7348 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:55.433702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:55.444132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321219ce-fc6c-422d-aa2e-e6d90bcc7348 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:55.451126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1daeb-c5e6-45e1-848e-7a05836f1a0b None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:55.451407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1daeb-c5e6-45e1-848e-7a05836f1a0b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:55.451689 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1daeb-c5e6-45e1-848e-7a05836f1a0b None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:55.453385 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 269/549] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:55.459889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cca7ba-9614-4b1c-86e9-4452f8f1320e None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:55.460452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cca7ba-9614-4b1c-86e9-4452f8f1320e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:55.460937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cca7ba-9614-4b1c-86e9-4452f8f1320e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:55.486962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-15cca7ba-9614-4b1c-86e9-4452f8f1320e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:55.488539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-15cca7ba-9614-4b1c-86e9-4452f8f1320e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:55.502549 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc5de0-a905-4541-af4c-372d2369c068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:55.503277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc5de0-a905-4541-af4c-372d2369c068 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:55.503570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc5de0-a905-4541-af4c-372d2369c068 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:55.503842 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc5de0-a905-4541-af4c-372d2369c068 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:55.504749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc5de0-a905-4541-af4c-372d2369c068 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:55.504997 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc5de0-a905-4541-af4c-372d2369c068 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:55.506447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc5de0-a905-4541-af4c-372d2369c068 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:55.545691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 281/550] 10.222.0.52 () {56 vars in 1135 bytes} [Thu May 19 13:40:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:55.545691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc5de0-a905-4541-af4c-372d2369c068 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:55.561294 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:55.592768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 270/551] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:55.671942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cca7ba-9614-4b1c-86e9-4452f8f1320e None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:55.676768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc5de0-a905-4541-af4c-372d2369c068 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:55.747915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d076b35-7423-4ca1-bdb1-2593ed31acc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:55.748891 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d076b35-7423-4ca1-bdb1-2593ed31acc7 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:55.749409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d076b35-7423-4ca1-bdb1-2593ed31acc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:55.749854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d076b35-7423-4ca1-bdb1-2593ed31acc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:55.794544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cca7ba-9614-4b1c-86e9-4452f8f1320e None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:55.819314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:55.864361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016c119-0e4b-4fa3-b234-e94cd23cd0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:55.865187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016c119-0e4b-4fa3-b234-e94cd23cd0a0 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:55.865477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016c119-0e4b-4fa3-b234-e94cd23cd0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:55.865758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016c119-0e4b-4fa3-b234-e94cd23cd0a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:55.887815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d076b35-7423-4ca1-bdb1-2593ed31acc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:55.889358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d076b35-7423-4ca1-bdb1-2593ed31acc7 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:55.892384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016c119-0e4b-4fa3-b234-e94cd23cd0a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-3016c119-0e4b-4fa3-b234-e94cd23cd0a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:55.903436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:55.904500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:55.905319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:55.905319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:55.905319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:55.905319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:55.905319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:55.905319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:55.905319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:55.905319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:55.905319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:55.905319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:55.905319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:55.905319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:55.905319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 271/552] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:40:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:55.906506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478cd95-7dd6-4f96-9ff5-d47cce738c1f None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:55.906603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478cd95-7dd6-4f96-9ff5-d47cce738c1f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:55.906992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478cd95-7dd6-4f96-9ff5-d47cce738c1f None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:55.913103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 272/553] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:55.913933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478cd95-7dd6-4f96-9ff5-d47cce738c1f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:55.979281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:56.022888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 282/554] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:40:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:40:56.024708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad399ced-4f56-4763-a1de-38f74e87c1ef None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.025086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad399ced-4f56-4763-a1de-38f74e87c1ef None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.025721 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad399ced-4f56-4763-a1de-38f74e87c1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.036519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478cd95-7dd6-4f96-9ff5-d47cce738c1f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:56.044261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ad399ced-4f56-4763-a1de-38f74e87c1ef None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:56.045472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ad399ced-4f56-4763-a1de-38f74e87c1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:56.075670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c04dfa-84c1-47d0-af91-4f7bb748b407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:56.076332 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c04dfa-84c1-47d0-af91-4f7bb748b407 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.076613 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c04dfa-84c1-47d0-af91-4f7bb748b407 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.076891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c04dfa-84c1-47d0-af91-4f7bb748b407 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.077652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c04dfa-84c1-47d0-af91-4f7bb748b407 None admin] RBAC: Adding query filter params (name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:56.077928 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c04dfa-84c1-47d0-af91-4f7bb748b407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:56.079851 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c04dfa-84c1-47d0-af91-4f7bb748b407 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:56.088422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 273/555] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:40:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:56.089205 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c04dfa-84c1-47d0-af91-4f7bb748b407 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:56.098658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:56.128066 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 283/556] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:56.129869 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df703948-4818-4406-aac9-a5ba50d048fb None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.130167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df703948-4818-4406-aac9-a5ba50d048fb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.130447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df703948-4818-4406-aac9-a5ba50d048fb None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.135743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 284/557] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:56.135743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df703948-4818-4406-aac9-a5ba50d048fb None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:56.208838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c04dfa-84c1-47d0-af91-4f7bb748b407 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:56.226060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:56.250761 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0150f-fc42-4759-a141-b35b7d06dde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:56.251747 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0150f-fc42-4759-a141-b35b7d06dde1 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.252198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0150f-fc42-4759-a141-b35b7d06dde1 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.252556 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0150f-fc42-4759-a141-b35b7d06dde1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.264366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0150f-fc42-4759-a141-b35b7d06dde1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:56.265817 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0150f-fc42-4759-a141-b35b7d06dde1 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:56.271022 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df703948-4818-4406-aac9-a5ba50d048fb None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-ccf0150f-fc42-4759-a141-b35b7d06dde1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:56.283663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:56.284604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:56.286112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:56.286112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:56.286112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:56.286112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:56.286112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 274/558] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:56.286800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fda5e-0fcb-4cf8-9684-fe09f7d7491d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.286992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fda5e-0fcb-4cf8-9684-fe09f7d7491d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.287153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fda5e-0fcb-4cf8-9684-fe09f7d7491d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.304137 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-b71fda5e-0fcb-4cf8-9684-fe09f7d7491d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:56.305304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b71fda5e-0fcb-4cf8-9684-fe09f7d7491d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:56.317775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205be3e-bc89-466c-8891-99c1c3545ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:56.318990 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205be3e-bc89-466c-8891-99c1c3545ebb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.319502 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205be3e-bc89-466c-8891-99c1c3545ebb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.320042 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205be3e-bc89-466c-8891-99c1c3545ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.331299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205be3e-bc89-466c-8891-99c1c3545ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:56.333153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205be3e-bc89-466c-8891-99c1c3545ebb None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-4205be3e-bc89-466c-8891-99c1c3545ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:56.352226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:56.353584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:56.354454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:56.354454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:56.354454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:56.354454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:56.356428 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 285/559] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:56.357634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205be3e-bc89-466c-8891-99c1c3545ebb None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:56.383035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 275/560] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:56.383035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fda5e-0fcb-4cf8-9684-fe09f7d7491d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:56.399603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:56.469053 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205be3e-bc89-466c-8891-99c1c3545ebb None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:56.478940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:56.485162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fda5e-0fcb-4cf8-9684-fe09f7d7491d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:56.503201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5db19f-25e8-469b-8572-f1865f56066b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:56.504610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5db19f-25e8-469b-8572-f1865f56066b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.504890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5db19f-25e8-469b-8572-f1865f56066b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.505167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5db19f-25e8-469b-8572-f1865f56066b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.515325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5db19f-25e8-469b-8572-f1865f56066b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:56.519459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31894a60-feec-468a-a9f5-a1909f66f6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:56.520145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31894a60-feec-468a-a9f5-a1909f66f6cf None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.520433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31894a60-feec-468a-a9f5-a1909f66f6cf None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.520713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31894a60-feec-468a-a9f5-a1909f66f6cf None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.521428 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31894a60-feec-468a-a9f5-a1909f66f6cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:56.521736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31894a60-feec-468a-a9f5-a1909f66f6cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:56.523132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31894a60-feec-468a-a9f5-a1909f66f6cf None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-0d5db19f-25e8-469b-8572-f1865f56066b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:56.523224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:56.524166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:56.524973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:56.524973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:56.524973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:56.524973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:56.524973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:56.524973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:56.524973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:56.524973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:56.524973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:56.524973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:56.524973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:56.524973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:56.524973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 286/561] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:40:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:56.525643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5db19f-25e8-469b-8572-f1865f56066b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:56.537529 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 276/562] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:56.538810 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357cfde-4378-420f-b0b8-1b19441915cd None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.539138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357cfde-4378-420f-b0b8-1b19441915cd None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.539516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357cfde-4378-420f-b0b8-1b19441915cd None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.541843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 277/563] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:40:56.542014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357cfde-4378-420f-b0b8-1b19441915cd None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:56.618453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:56.629711 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5db19f-25e8-469b-8572-f1865f56066b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:56.642616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357cfde-4378-420f-b0b8-1b19441915cd None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:56.663153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196349a7-ebeb-49ad-8a05-18b3b3416ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:56.663823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196349a7-ebeb-49ad-8a05-18b3b3416ec2 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.664111 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196349a7-ebeb-49ad-8a05-18b3b3416ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.664393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196349a7-ebeb-49ad-8a05-18b3b3416ec2 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.665119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196349a7-ebeb-49ad-8a05-18b3b3416ec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:56.665431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196349a7-ebeb-49ad-8a05-18b3b3416ec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:56.666799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196349a7-ebeb-49ad-8a05-18b3b3416ec2 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:56.675923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf479087-b773-4810-9a50-df6fbc4f556c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:56.676703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf479087-b773-4810-9a50-df6fbc4f556c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.676981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf479087-b773-4810-9a50-df6fbc4f556c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.677249 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf479087-b773-4810-9a50-df6fbc4f556c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.678575 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 287/564] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:56.679404 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196349a7-ebeb-49ad-8a05-18b3b3416ec2 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:56.688454 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf479087-b773-4810-9a50-df6fbc4f556c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-cf479087-b773-4810-9a50-df6fbc4f556c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:56.696627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:56.697578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:56.698411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:56.698411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:56.698411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:56.698411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:56.698411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:56.698411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:56.698411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:56.698411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:56.698411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:56.698411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:56.698411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:56.698411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:56.698411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 278/565] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:40:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:56.699630 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf479087-b773-4810-9a50-df6fbc4f556c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:56.720952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:56.777178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196349a7-ebeb-49ad-8a05-18b3b3416ec2 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:56.801860 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf479087-b773-4810-9a50-df6fbc4f556c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:56.810946 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa6c8a-e3a3-40b1-afbc-1d42e4ee0794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:56.811611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa6c8a-e3a3-40b1-afbc-1d42e4ee0794 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.811906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa6c8a-e3a3-40b1-afbc-1d42e4ee0794 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.812181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa6c8a-e3a3-40b1-afbc-1d42e4ee0794 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.812971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa6c8a-e3a3-40b1-afbc-1d42e4ee0794 None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:56.813274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa6c8a-e3a3-40b1-afbc-1d42e4ee0794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:56.814628 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa6c8a-e3a3-40b1-afbc-1d42e4ee0794 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:56.823402 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 288/566] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:40:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:56.824663 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52640de9-3117-41b9-a982-d98ed264fe72 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.824957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52640de9-3117-41b9-a982-d98ed264fe72 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.825241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52640de9-3117-41b9-a982-d98ed264fe72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.839764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68da73c-b08e-4092-adf0-dcc08ab5aab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:56.840548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68da73c-b08e-4092-adf0-dcc08ab5aab3 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:56.840829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68da73c-b08e-4092-adf0-dcc08ab5aab3 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:56.841097 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68da73c-b08e-4092-adf0-dcc08ab5aab3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:56.841512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-52640de9-3117-41b9-a982-d98ed264fe72 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:40:56.842717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-52640de9-3117-41b9-a982-d98ed264fe72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:56.847992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68da73c-b08e-4092-adf0-dcc08ab5aab3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:56.849410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68da73c-b08e-4092-adf0-dcc08ab5aab3 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:56.850079 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-c68da73c-b08e-4092-adf0-dcc08ab5aab3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:40:56.863378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:56.864452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:56.865535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:40:56.865535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:40:56.865535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:40:56.865535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:40:56.865535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:40:56.865535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:40:56.865535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:56.865535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:56.865535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 279/567] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:56.865535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68da73c-b08e-4092-adf0-dcc08ab5aab3 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:56.915264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 289/568] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:40:56.916096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52640de9-3117-41b9-a982-d98ed264fe72 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:56.942379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:56.965933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68da73c-b08e-4092-adf0-dcc08ab5aab3 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:56.999545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c54e74-378c-4257-865a-e2b7345624a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:57.000338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c54e74-378c-4257-865a-e2b7345624a0 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:57.000623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c54e74-378c-4257-865a-e2b7345624a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:57.000905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c54e74-378c-4257-865a-e2b7345624a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:57.008614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c54e74-378c-4257-865a-e2b7345624a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:57.010054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c54e74-378c-4257-865a-e2b7345624a0 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:57.021992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52640de9-3117-41b9-a982-d98ed264fe72 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-71c54e74-378c-4257-865a-e2b7345624a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:40:57.024355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:57.025318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:57.026255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:57.026255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:57.026255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:40:57.026255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:40:57.026255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:40:57.026255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:40:57.026255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:40:57.026255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:40:57.026255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:40:57.026255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:57.026255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 280/569] 10.222.0.52 () {56 vars in 1093 bytes} [Thu May 19 13:40:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:57.026878 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c54e74-378c-4257-865a-e2b7345624a0 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:57.056097 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deee18-e80b-4829-8e19-2b2baf645345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:57.056688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deee18-e80b-4829-8e19-2b2baf645345 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:57.056980 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deee18-e80b-4829-8e19-2b2baf645345 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:57.057255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deee18-e80b-4829-8e19-2b2baf645345 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:57.058004 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24deee18-e80b-4829-8e19-2b2baf645345 None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:57.058308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24deee18-e80b-4829-8e19-2b2baf645345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:57.059688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24deee18-e80b-4829-8e19-2b2baf645345 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:57.068081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 290/570] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:40:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:57.068871 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deee18-e80b-4829-8e19-2b2baf645345 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:57.073829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:57.129884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c54e74-378c-4257-865a-e2b7345624a0 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:57.163289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a75d54-a3e2-4ca5-89d7-feded54d3a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:57.164072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a75d54-a3e2-4ca5-89d7-feded54d3a46 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:57.164358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a75d54-a3e2-4ca5-89d7-feded54d3a46 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:57.164631 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a75d54-a3e2-4ca5-89d7-feded54d3a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:57.167384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:57.169986 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deee18-e80b-4829-8e19-2b2baf645345 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:57.176574 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a75d54-a3e2-4ca5-89d7-feded54d3a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:57.178006 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a75d54-a3e2-4ca5-89d7-feded54d3a46 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-05a75d54-a3e2-4ca5-89d7-feded54d3a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:57.196260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:57.197143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:57.198029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:57.198029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:57.198029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:57.198029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:57.198029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 281/571] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:57.198299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a75d54-a3e2-4ca5-89d7-feded54d3a46 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:57.207525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1eb435-434c-4df5-a1e7-a0ab6467ebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:57.208187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1eb435-434c-4df5-a1e7-a0ab6467ebec None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:57.208472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1eb435-434c-4df5-a1e7-a0ab6467ebec None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:57.208751 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1eb435-434c-4df5-a1e7-a0ab6467ebec None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:57.209535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1eb435-434c-4df5-a1e7-a0ab6467ebec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:57.209849 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1eb435-434c-4df5-a1e7-a0ab6467ebec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:57.211339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1eb435-434c-4df5-a1e7-a0ab6467ebec None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:57.224419 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 291/572] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:40:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:57.225223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1eb435-434c-4df5-a1e7-a0ab6467ebec None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:57.285872 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:57.300230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a75d54-a3e2-4ca5-89d7-feded54d3a46 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:57.323252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1eb435-434c-4df5-a1e7-a0ab6467ebec None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:57.333714 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5c3a4-0aa8-49d4-b873-85e08e86d3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:57.334495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5c3a4-0aa8-49d4-b873-85e08e86d3fc None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:57.334781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5c3a4-0aa8-49d4-b873-85e08e86d3fc None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:57.335087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5c3a4-0aa8-49d4-b873-85e08e86d3fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:57.344828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5c3a4-0aa8-49d4-b873-85e08e86d3fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-50e5c3a4-0aa8-49d4-b873-85e08e86d3fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:40:57.352404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:57.353313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:57.354316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:57.354316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:57.354316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:57.354316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:57.354316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:40:57.354316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:40:57.354316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:40:57.354316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:40:57.354316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:40:57.354316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:40:57.354316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:40:57.354316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:57.354316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 282/573] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:40:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:57.355295 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5c3a4-0aa8-49d4-b873-85e08e86d3fc None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:57.361906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891ca6b-b9ad-424a-b619-28648a789226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:57.362567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891ca6b-b9ad-424a-b619-28648a789226 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:57.362922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891ca6b-b9ad-424a-b619-28648a789226 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:57.363180 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891ca6b-b9ad-424a-b619-28648a789226 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:57.364009 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891ca6b-b9ad-424a-b619-28648a789226 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:57.364313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891ca6b-b9ad-424a-b619-28648a789226 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:57.365729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891ca6b-b9ad-424a-b619-28648a789226 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:57.383214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 292/574] 10.222.0.52 () {56 vars in 1133 bytes} [Thu May 19 13:40:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:57.384003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891ca6b-b9ad-424a-b619-28648a789226 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:57.409500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:57.462214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5c3a4-0aa8-49d4-b873-85e08e86d3fc None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:57.483551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891ca6b-b9ad-424a-b619-28648a789226 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:57.495563 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3090657-f1ab-49bb-b55a-f16b66e5e792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:57.496351 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3090657-f1ab-49bb-b55a-f16b66e5e792 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:57.496887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3090657-f1ab-49bb-b55a-f16b66e5e792 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:57.497178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3090657-f1ab-49bb-b55a-f16b66e5e792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:57.507158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:57.508160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3090657-f1ab-49bb-b55a-f16b66e5e792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:57.509765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3090657-f1ab-49bb-b55a-f16b66e5e792 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:57.518220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3dea7-d925-4765-9891-9362f8f63f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:57.518904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3dea7-d925-4765-9891-9362f8f63f7c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:57.520464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3dea7-d925-4765-9891-9362f8f63f7c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:57.520752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3dea7-d925-4765-9891-9362f8f63f7c None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:57.521473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3dea7-d925-4765-9891-9362f8f63f7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:57.521778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3dea7-d925-4765-9891-9362f8f63f7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:57.523142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3dea7-d925-4765-9891-9362f8f63f7c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-f3090657-f1ab-49bb-b55a-f16b66e5e792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:57.527260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:57.528112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:57.528981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:57.528981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:57.528981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:57.528981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:57.528981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 283/575] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:57.529707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3090657-f1ab-49bb-b55a-f16b66e5e792 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:57.538111 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 293/576] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:57.538951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3dea7-d925-4765-9891-9362f8f63f7c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:57.627249 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:57.630905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3090657-f1ab-49bb-b55a-f16b66e5e792 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:57.636085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3dea7-d925-4765-9891-9362f8f63f7c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:57.665585 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1b1e9-dc7c-4dcc-8cbd-2a51707a3b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:57.666238 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1b1e9-dc7c-4dcc-8cbd-2a51707a3b94 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:57.666525 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1b1e9-dc7c-4dcc-8cbd-2a51707a3b94 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:57.666802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1b1e9-dc7c-4dcc-8cbd-2a51707a3b94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:57.667664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1b1e9-dc7c-4dcc-8cbd-2a51707a3b94 None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=8554ff40bbcb4fbb801d854d6527a6d2) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:57.667978 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1b1e9-dc7c-4dcc-8cbd-2a51707a3b94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:57.668846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258230a-2582-4389-912b-c3ab0605e505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:57.669402 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1b1e9-dc7c-4dcc-8cbd-2a51707a3b94 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:57.669512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258230a-2582-4389-912b-c3ab0605e505 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:57.669773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258230a-2582-4389-912b-c3ab0605e505 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:57.670050 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258230a-2582-4389-912b-c3ab0605e505 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:57.670799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258230a-2582-4389-912b-c3ab0605e505 None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:57.671136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258230a-2582-4389-912b-c3ab0605e505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:57.672469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258230a-2582-4389-912b-c3ab0605e505 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:57.674745 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 284/577] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:40:57 2022] GET /identity/v3/role_assignments?user.id=8554ff40bbcb4fbb801d854d6527a6d2&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:57.676104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1b1e9-dc7c-4dcc-8cbd-2a51707a3b94 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:57.686023 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 294/578] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:40:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:57.686023 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258230a-2582-4389-912b-c3ab0605e505 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:57.741224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:57.803558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258230a-2582-4389-912b-c3ab0605e505 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:57.803910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1b1e9-dc7c-4dcc-8cbd-2a51707a3b94 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:57.841139 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638387f-6883-40a3-97c2-fc62cf48f984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:57.842035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638387f-6883-40a3-97c2-fc62cf48f984 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:57.842356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638387f-6883-40a3-97c2-fc62cf48f984 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:57.842638 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638387f-6883-40a3-97c2-fc62cf48f984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:57.845695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079e8dd-d9ae-421a-955f-a1f9f973f1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:57.846370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e8dd-d9ae-421a-955f-a1f9f973f1ae None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:57.846657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e8dd-d9ae-421a-955f-a1f9f973f1ae None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:57.847023 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e8dd-d9ae-421a-955f-a1f9f973f1ae None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:57.848292 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079e8dd-d9ae-421a-955f-a1f9f973f1ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:57.848604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079e8dd-d9ae-421a-955f-a1f9f973f1ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:57.849977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079e8dd-d9ae-421a-955f-a1f9f973f1ae None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:57.855692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638387f-6883-40a3-97c2-fc62cf48f984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:57.857131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638387f-6883-40a3-97c2-fc62cf48f984 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:57.863325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 295/579] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:57.864228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079e8dd-d9ae-421a-955f-a1f9f973f1ae None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-6638387f-6883-40a3-97c2-fc62cf48f984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:57.878194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:57.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:57.880022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:57.880022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:57.880022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:57.880022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:57.880022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 285/580] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:57.880851 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638387f-6883-40a3-97c2-fc62cf48f984 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:57.902289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:57.974060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079e8dd-d9ae-421a-955f-a1f9f973f1ae None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:57.977213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:57.993987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638387f-6883-40a3-97c2-fc62cf48f984 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:58.014189 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceac474-a6a6-4511-a226-a9acef6fd7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:58.015020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceac474-a6a6-4511-a226-a9acef6fd7b1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:58.015317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceac474-a6a6-4511-a226-a9acef6fd7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:58.015596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceac474-a6a6-4511-a226-a9acef6fd7b1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:58.023508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceac474-a6a6-4511-a226-a9acef6fd7b1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:58.024944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceac474-a6a6-4511-a226-a9acef6fd7b1 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:58.029787 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911cf19-2bf0-44a5-bee1-13486f711e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:58.030799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911cf19-2bf0-44a5-bee1-13486f711e62 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:58.031264 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911cf19-2bf0-44a5-bee1-13486f711e62 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:58.031708 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911cf19-2bf0-44a5-bee1-13486f711e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-5ceac474-a6a6-4511-a226-a9acef6fd7b1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:40:58.040992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:58.042131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:58.043221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:58.043221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:58.043221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:40:58.043221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:40:58.043221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:40:58.043221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:40:58.043221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:40:58.043221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:40:58.043221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:40:58.043221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:58.043221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 296/581] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:58.043810 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceac474-a6a6-4511-a226-a9acef6fd7b1 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:58.049065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911cf19-2bf0-44a5-bee1-13486f711e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:58.051084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911cf19-2bf0-44a5-bee1-13486f711e62 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-9911cf19-2bf0-44a5-bee1-13486f711e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:58.069790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:58.070723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:58.071657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:58.071657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:58.071657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:58.071657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:58.075833 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 286/582] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:58.076551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911cf19-2bf0-44a5-bee1-13486f711e62 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:58.149411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:58.150662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceac474-a6a6-4511-a226-a9acef6fd7b1 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:58.188562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc693f-c211-4fd9-885c-7fb6efc737b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:58.189479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911cf19-2bf0-44a5-bee1-13486f711e62 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:58.189970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc693f-c211-4fd9-885c-7fb6efc737b7 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:58.190462 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc693f-c211-4fd9-885c-7fb6efc737b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:58.191021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc693f-c211-4fd9-885c-7fb6efc737b7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:58.198470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc693f-c211-4fd9-885c-7fb6efc737b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:58.199963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc693f-c211-4fd9-885c-7fb6efc737b7 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-1dfc693f-c211-4fd9-885c-7fb6efc737b7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:40:58.215209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:58.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:58.217126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:58.217126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:58.217126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:40:58.217126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:40:58.217126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:40:58.217126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:40:58.217126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:40:58.217126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:40:58.217126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 13:40:58.217126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:58.217126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 297/583] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:40:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:58.218364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc693f-c211-4fd9-885c-7fb6efc737b7 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:58.230176 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38110778-9357-41c2-bb33-4372408c328c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:58.230505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38110778-9357-41c2-bb33-4372408c328c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:58.230784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38110778-9357-41c2-bb33-4372408c328c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:58.231106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38110778-9357-41c2-bb33-4372408c328c None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:58.231824 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38110778-9357-41c2-bb33-4372408c328c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:58.232131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38110778-9357-41c2-bb33-4372408c328c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:58.233589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38110778-9357-41c2-bb33-4372408c328c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:58.248132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 287/584] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:58.248946 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38110778-9357-41c2-bb33-4372408c328c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:58.254562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:58.354982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc693f-c211-4fd9-885c-7fb6efc737b7 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:58.376256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38110778-9357-41c2-bb33-4372408c328c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:58.397962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d34558-bb93-4222-8e90-a6ce54353b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:58.400260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d34558-bb93-4222-8e90-a6ce54353b48 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:58.400556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d34558-bb93-4222-8e90-a6ce54353b48 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:58.400838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d34558-bb93-4222-8e90-a6ce54353b48 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:58.401692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d34558-bb93-4222-8e90-a6ce54353b48 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:58.402007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d34558-bb93-4222-8e90-a6ce54353b48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:58.403511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d34558-bb93-4222-8e90-a6ce54353b48 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:58.408826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:58.420059 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa115998-4639-4fc5-b5f5-35a953f56c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:58.420587 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa115998-4639-4fc5-b5f5-35a953f56c3c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:58.420875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa115998-4639-4fc5-b5f5-35a953f56c3c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:58.421153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa115998-4639-4fc5-b5f5-35a953f56c3c None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:58.421871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa115998-4639-4fc5-b5f5-35a953f56c3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:58.422178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa115998-4639-4fc5-b5f5-35a953f56c3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:58.423662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 298/585] 10.222.0.52 () {56 vars in 1137 bytes} [Thu May 19 13:40:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:58.424334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d34558-bb93-4222-8e90-a6ce54353b48 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:58.428991 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa115998-4639-4fc5-b5f5-35a953f56c3c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:58.446648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 288/586] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:58.446648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa115998-4639-4fc5-b5f5-35a953f56c3c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:58.516542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:58.536960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d34558-bb93-4222-8e90-a6ce54353b48 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:58.557278 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa115998-4639-4fc5-b5f5-35a953f56c3c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:58.573465 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5eb349-e069-43f0-8da0-4a71499fea33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:58.574126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eb349-e069-43f0-8da0-4a71499fea33 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:58.574420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eb349-e069-43f0-8da0-4a71499fea33 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:58.574699 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eb349-e069-43f0-8da0-4a71499fea33 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:58.575563 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eb349-e069-43f0-8da0-4a71499fea33 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:58.575876 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eb349-e069-43f0-8da0-4a71499fea33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:58.577307 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eb349-e069-43f0-8da0-4a71499fea33 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:58.595194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 299/587] 10.222.0.52 () {56 vars in 1137 bytes} [Thu May 19 13:40:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:58.595718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ede26-aa6c-4425-8402-c6264e7f07da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:58.595999 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5eb349-e069-43f0-8da0-4a71499fea33 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:58.597125 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ede26-aa6c-4425-8402-c6264e7f07da None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:58.597715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ede26-aa6c-4425-8402-c6264e7f07da None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:58.598187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ede26-aa6c-4425-8402-c6264e7f07da None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:58.607011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ede26-aa6c-4425-8402-c6264e7f07da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:58.608569 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ede26-aa6c-4425-8402-c6264e7f07da None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-833ede26-aa6c-4425-8402-c6264e7f07da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:40:58.624240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:58.625221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:58.626154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:40:58.626154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:40:58.626154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:40:58.626154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:40:58.626154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:40:58.626154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:40:58.626154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:58.626154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:58.626154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 289/588] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:58.626672 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ede26-aa6c-4425-8402-c6264e7f07da None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:58.684125 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:58.715042 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5eb349-e069-43f0-8da0-4a71499fea33 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:58.738804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ede26-aa6c-4425-8402-c6264e7f07da None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:58.750364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2b6f8-0baf-421c-8004-fa7650f2014c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:58.751183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2b6f8-0baf-421c-8004-fa7650f2014c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:58.751467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2b6f8-0baf-421c-8004-fa7650f2014c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:58.751743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2b6f8-0baf-421c-8004-fa7650f2014c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:58.761136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:58.763475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2b6f8-0baf-421c-8004-fa7650f2014c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:58.765043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2b6f8-0baf-421c-8004-fa7650f2014c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:58.778485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb58406-ba6b-40c0-8b27-a1684d417084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:58.779683 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb58406-ba6b-40c0-8b27-a1684d417084 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:58.780119 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb58406-ba6b-40c0-8b27-a1684d417084 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:58.780539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb58406-ba6b-40c0-8b27-a1684d417084 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-f1a2b6f8-0baf-421c-8004-fa7650f2014c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:58.784115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:58.785327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:58.786304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:58.786304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:58.786304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:58.786304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:58.786304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 300/589] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:58.786597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2b6f8-0baf-421c-8004-fa7650f2014c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:58.792512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb58406-ba6b-40c0-8b27-a1684d417084 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:58.794518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb58406-ba6b-40c0-8b27-a1684d417084 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-6eb58406-ba6b-40c0-8b27-a1684d417084 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:40:58.811329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:58.812358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:58.813290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:58.813290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:58.813290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:40:58.813290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:40:58.813290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:40:58.813290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:40:58.813290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:40:58.813290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:40:58.813290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 13:40:58.813290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:58.813290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 290/590] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:40:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:58.814428 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb58406-ba6b-40c0-8b27-a1684d417084 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:58.900608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2b6f8-0baf-421c-8004-fa7650f2014c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:58.925739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb58406-ba6b-40c0-8b27-a1684d417084 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:58.935498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:58.935918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e61ecf-ce50-4a96-b9f2-8c1169a00298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:58.936807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e61ecf-ce50-4a96-b9f2-8c1169a00298 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:58.937155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e61ecf-ce50-4a96-b9f2-8c1169a00298 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:58.937422 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e61ecf-ce50-4a96-b9f2-8c1169a00298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:58.952668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e61ecf-ce50-4a96-b9f2-8c1169a00298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:58.954108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e61ecf-ce50-4a96-b9f2-8c1169a00298 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:58.970486 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ddfcb-5bd3-4ebf-993c-cebe74a244a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:58.970741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ddfcb-5bd3-4ebf-993c-cebe74a244a1 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:58.970830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ddfcb-5bd3-4ebf-993c-cebe74a244a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:58.971160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ddfcb-5bd3-4ebf-993c-cebe74a244a1 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:58.972974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ddfcb-5bd3-4ebf-993c-cebe74a244a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:58.972974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ddfcb-5bd3-4ebf-993c-cebe74a244a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:58.974190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ddfcb-5bd3-4ebf-993c-cebe74a244a1 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-59e61ecf-ce50-4a96-b9f2-8c1169a00298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:58.975873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:58.976808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:58.977728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:58.977728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:58.977728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:58.977728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:58.981710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 301/591] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:58.982545 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e61ecf-ce50-4a96-b9f2-8c1169a00298 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:58.993703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 291/592] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:40:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:58.994544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ddfcb-5bd3-4ebf-993c-cebe74a244a1 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:59.039645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:59.101182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e61ecf-ce50-4a96-b9f2-8c1169a00298 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:59.103947 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ddfcb-5bd3-4ebf-993c-cebe74a244a1 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:59.137200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608743e-cdb0-4e8f-baba-2feb7ce6d94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:59.137400 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f542fb8-b764-4222-a166-1a0353162939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:59.138028 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f542fb8-b764-4222-a166-1a0353162939 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:59.138310 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f542fb8-b764-4222-a166-1a0353162939 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:59.138587 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f542fb8-b764-4222-a166-1a0353162939 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:59.139087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608743e-cdb0-4e8f-baba-2feb7ce6d94d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:59.139437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f542fb8-b764-4222-a166-1a0353162939 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:59.139742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f542fb8-b764-4222-a166-1a0353162939 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:59.140019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608743e-cdb0-4e8f-baba-2feb7ce6d94d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:59.140503 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608743e-cdb0-4e8f-baba-2feb7ce6d94d None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:59.141157 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f542fb8-b764-4222-a166-1a0353162939 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:59.141641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608743e-cdb0-4e8f-baba-2feb7ce6d94d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:59.142154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608743e-cdb0-4e8f-baba-2feb7ce6d94d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:59.144541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608743e-cdb0-4e8f-baba-2feb7ce6d94d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:59.158035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 302/593] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:59.159174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608743e-cdb0-4e8f-baba-2feb7ce6d94d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:59.159394 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 292/594] 10.222.0.52 () {56 vars in 1139 bytes} [Thu May 19 13:40:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:59.160440 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f542fb8-b764-4222-a166-1a0353162939 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:59.194827 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:59.271099 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f542fb8-b764-4222-a166-1a0353162939 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:59.273499 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608743e-cdb0-4e8f-baba-2feb7ce6d94d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:59.275841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:59.309584 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a162193-8b29-4dc1-a0a5-e7c810d28b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:59.310577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a162193-8b29-4dc1-a0a5-e7c810d28b12 None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:59.310881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a162193-8b29-4dc1-a0a5-e7c810d28b12 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:59.311176 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a162193-8b29-4dc1-a0a5-e7c810d28b12 None admin] PATH_INFO: `/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/4f6093d0c1fc41d98c58bace81e3d8e5/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:59.312261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dd7c0-5a38-43c7-8e43-f35b19844d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:59.312951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dd7c0-5a38-43c7-8e43-f35b19844d7c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:59.313236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dd7c0-5a38-43c7-8e43-f35b19844d7c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:59.313507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dd7c0-5a38-43c7-8e43-f35b19844d7c None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:59.314223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787dd7c0-5a38-43c7-8e43-f35b19844d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:59.314525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787dd7c0-5a38-43c7-8e43-f35b19844d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:59.315974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787dd7c0-5a38-43c7-8e43-f35b19844d7c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:59.328260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a162193-8b29-4dc1-a0a5-e7c810d28b12 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fc6fd42a31942688f570f1ce86be510, user_id=4f6093d0c1fc41d98c58bace81e3d8e5, role_id=c0cbd2d231594ea2bf7bb013fbf9ad1f)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:59.328730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 303/595] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:59.329847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dd7c0-5a38-43c7-8e43-f35b19844d7c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:59.332948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a162193-8b29-4dc1-a0a5-e7c810d28b12 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:59.358962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 293/596] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:40:59 2022] PUT /identity/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/4f6093d0c1fc41d98c58bace81e3d8e5/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:40:59.360005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a162193-8b29-4dc1-a0a5-e7c810d28b12 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:59.483961 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dd7c0-5a38-43c7-8e43-f35b19844d7c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:59.493618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:59.530816 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a162193-8b29-4dc1-a0a5-e7c810d28b12 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:59.531186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04179e-17e5-47e5-bbc1-a4a02f5e0779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:59.531976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04179e-17e5-47e5-bbc1-a4a02f5e0779 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:59.532255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04179e-17e5-47e5-bbc1-a4a02f5e0779 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:59.532719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04179e-17e5-47e5-bbc1-a4a02f5e0779 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:59.542123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04179e-17e5-47e5-bbc1-a4a02f5e0779 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:59.544055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04179e-17e5-47e5-bbc1-a4a02f5e0779 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-0f04179e-17e5-47e5-bbc1-a4a02f5e0779 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:40:59.564797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:59.565834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:59.567338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:40:59.567338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:40:59.567338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:40:59.567338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:40:59.567338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:40:59.567338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:40:59.567338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:59.567338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:59.567338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 304/597] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:59.568391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04179e-17e5-47e5-bbc1-a4a02f5e0779 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:59.581232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8218a23-fcbd-4577-9dff-ed83c73755e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:59.582235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8218a23-fcbd-4577-9dff-ed83c73755e8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:59.582235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8218a23-fcbd-4577-9dff-ed83c73755e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:59.582583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8218a23-fcbd-4577-9dff-ed83c73755e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:59.594426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8218a23-fcbd-4577-9dff-ed83c73755e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:59.596559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8218a23-fcbd-4577-9dff-ed83c73755e8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:59.601625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-d8218a23-fcbd-4577-9dff-ed83c73755e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:40:59.616047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:59.617088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:59.617965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:40:59.617965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:40:59.617965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:40:59.617965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:40:59.617965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 294/598] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:59.618444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8218a23-fcbd-4577-9dff-ed83c73755e8 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:59.701943 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04179e-17e5-47e5-bbc1-a4a02f5e0779 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:59.743912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb206f-728c-45c2-92bd-0c59610026ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:59.744627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb206f-728c-45c2-92bd-0c59610026ea None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:59.744913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb206f-728c-45c2-92bd-0c59610026ea None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:59.745224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb206f-728c-45c2-92bd-0c59610026ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:59.753535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb206f-728c-45c2-92bd-0c59610026ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:59.755433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb206f-728c-45c2-92bd-0c59610026ea None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:59.759000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8218a23-fcbd-4577-9dff-ed83c73755e8 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-08bb206f-728c-45c2-92bd-0c59610026ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:40:59.770991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:40:59.771914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:40:59.772815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:40:59.772815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:40:59.772815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:40:59.772815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:40:59.772815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:40:59.772815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:40:59.772815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:40:59.772815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:40:59.772815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 305/599] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:40:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:40:59.773297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb206f-728c-45c2-92bd-0c59610026ea None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:59.793327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:59.805755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28ba55-3470-471c-900d-9e19075e7a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:59.807043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28ba55-3470-471c-900d-9e19075e7a0d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:59.808085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28ba55-3470-471c-900d-9e19075e7a0d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:59.808574 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28ba55-3470-471c-900d-9e19075e7a0d None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:59.809849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28ba55-3470-471c-900d-9e19075e7a0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:59.810618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28ba55-3470-471c-900d-9e19075e7a0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:59.813043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28ba55-3470-471c-900d-9e19075e7a0d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:59.830244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 295/600] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:40:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:59.831770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28ba55-3470-471c-900d-9e19075e7a0d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:59.883234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:59.903897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb206f-728c-45c2-92bd-0c59610026ea None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:59.941548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceedda6-93e6-49dd-9c98-e5f964c7cc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:59.942224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceedda6-93e6-49dd-9c98-e5f964c7cc4f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:59.942517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceedda6-93e6-49dd-9c98-e5f964c7cc4f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:59.942799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceedda6-93e6-49dd-9c98-e5f964c7cc4f None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:59.943559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceedda6-93e6-49dd-9c98-e5f964c7cc4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:40:59.943870 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceedda6-93e6-49dd-9c98-e5f964c7cc4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:59.945240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceedda6-93e6-49dd-9c98-e5f964c7cc4f None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:59.958347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 306/601] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:40:59.959200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceedda6-93e6-49dd-9c98-e5f964c7cc4f None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:59.963234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28ba55-3470-471c-900d-9e19075e7a0d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:59.999359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa01593-ee23-4570-9bee-ae628287628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:59.999941 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa01593-ee23-4570-9bee-ae628287628d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:00.000232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa01593-ee23-4570-9bee-ae628287628d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:00.000512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa01593-ee23-4570-9bee-ae628287628d None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:00.001329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa01593-ee23-4570-9bee-ae628287628d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:00.001641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa01593-ee23-4570-9bee-ae628287628d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:00.003105 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa01593-ee23-4570-9bee-ae628287628d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:00.017601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 296/602] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:40:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:00.023992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614fd7b-48c1-4ac6-860d-040369aae058 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:00.024294 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614fd7b-48c1-4ac6-860d-040369aae058 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:00.024574 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614fd7b-48c1-4ac6-860d-040369aae058 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:00.029728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 297/603] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:00.029728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614fd7b-48c1-4ac6-860d-040369aae058 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:00.071530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceedda6-93e6-49dd-9c98-e5f964c7cc4f None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:00.083735 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:00.112098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14700478-1dd6-419c-bd0f-b9ff59b21e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:00.114072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14700478-1dd6-419c-bd0f-b9ff59b21e02 None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:00.114359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14700478-1dd6-419c-bd0f-b9ff59b21e02 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:00.115074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14700478-1dd6-419c-bd0f-b9ff59b21e02 None admin] PATH_INFO: `/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/1ed0973226a2446ebf7c0cc3ee04e446/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:00.131374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14700478-1dd6-419c-bd0f-b9ff59b21e02 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fc6fd42a31942688f570f1ce86be510, user_id=1ed0973226a2446ebf7c0cc3ee04e446, role_id=c0cbd2d231594ea2bf7bb013fbf9ad1f)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:00.133086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14700478-1dd6-419c-bd0f-b9ff59b21e02 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:00.135665 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:00.141073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614fd7b-48c1-4ac6-860d-040369aae058 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:00.155251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 307/604] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:40:59 2022] PUT /identity/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/1ed0973226a2446ebf7c0cc3ee04e446/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:41:00.155251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14700478-1dd6-419c-bd0f-b9ff59b21e02 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:00.187503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8b6ee-6ce5-4308-a9b5-2dc7d60ee19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:00.188314 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b6ee-6ce5-4308-a9b5-2dc7d60ee19d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:00.188599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b6ee-6ce5-4308-a9b5-2dc7d60ee19d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:00.188876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b6ee-6ce5-4308-a9b5-2dc7d60ee19d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:00.197443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8b6ee-6ce5-4308-a9b5-2dc7d60ee19d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:00.198975 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8b6ee-6ce5-4308-a9b5-2dc7d60ee19d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-e2a8b6ee-6ce5-4308-a9b5-2dc7d60ee19d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:00.215015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:00.216256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:00.217240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:00.217240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:00.217240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:00.217240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:00.217240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:00.217240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:00.217240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:00.217240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:00.217240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 298/605] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:00.218093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dd814-1e26-4038-9881-4483bbf483e7 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:00.218386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dd814-1e26-4038-9881-4483bbf483e7 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:00.218722 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dd814-1e26-4038-9881-4483bbf483e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:00.241548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-c45dd814-1e26-4038-9881-4483bbf483e7 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:00.242918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c45dd814-1e26-4038-9881-4483bbf483e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:00.296543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14700478-1dd6-419c-bd0f-b9ff59b21e02 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:00.338239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96c3a8-9f86-40c3-a877-b0840e98ba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:00.339194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96c3a8-9f86-40c3-a877-b0840e98ba41 None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:00.339480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96c3a8-9f86-40c3-a877-b0840e98ba41 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:00.339755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96c3a8-9f86-40c3-a877-b0840e98ba41 None admin] PATH_INFO: `/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/666ade4ce1fe447487998d086779389b/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:00.344792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 299/606] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:41:00.345944 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dd814-1e26-4038-9881-4483bbf483e7 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:00.357249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96c3a8-9f86-40c3-a877-b0840e98ba41 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fc6fd42a31942688f570f1ce86be510, user_id=666ade4ce1fe447487998d086779389b, role_id=c0cbd2d231594ea2bf7bb013fbf9ad1f)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:00.358776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96c3a8-9f86-40c3-a877-b0840e98ba41 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:00.387855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 308/607] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:41:00 2022] PUT /identity/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/666ade4ce1fe447487998d086779389b/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:41:00.387855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96c3a8-9f86-40c3-a877-b0840e98ba41 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:00.415425 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:00.452111 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:00.511436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dd814-1e26-4038-9881-4483bbf483e7 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:00.534068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96c3a8-9f86-40c3-a877-b0840e98ba41 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:00.556250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca44692-79a3-491c-961e-1cc1589dde0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:00.557628 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca44692-79a3-491c-961e-1cc1589dde0a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:00.558135 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca44692-79a3-491c-961e-1cc1589dde0a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:00.558614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca44692-79a3-491c-961e-1cc1589dde0a None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:00.559679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca44692-79a3-491c-961e-1cc1589dde0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:00.560243 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca44692-79a3-491c-961e-1cc1589dde0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:00.561958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca44692-79a3-491c-961e-1cc1589dde0a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:00.577386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 300/608] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:00.583959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca44692-79a3-491c-961e-1cc1589dde0a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:00.611785 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd2270-198c-400e-82ba-6b5e2ad6e886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:00.613131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd2270-198c-400e-82ba-6b5e2ad6e886 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:00.613431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd2270-198c-400e-82ba-6b5e2ad6e886 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:00.613916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd2270-198c-400e-82ba-6b5e2ad6e886 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:00.626140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd2270-198c-400e-82ba-6b5e2ad6e886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-55cd2270-198c-400e-82ba-6b5e2ad6e886 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:00.636977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:00.637996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:00.638965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:00.638965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:00.638965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:00.638965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:00.638965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:00.638965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:00.638965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:00.638965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:00.638965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:00.638965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:00.638965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:00.638965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:00.638965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 309/609] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:41:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:00.645976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd2270-198c-400e-82ba-6b5e2ad6e886 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:00.722567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:00.727909 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca44692-79a3-491c-961e-1cc1589dde0a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:00.793229 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7f5fa-90fd-41cb-ac93-32b762ab5c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:00.794153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7f5fa-90fd-41cb-ac93-32b762ab5c3a None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:00.794477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7f5fa-90fd-41cb-ac93-32b762ab5c3a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:00.794830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7f5fa-90fd-41cb-ac93-32b762ab5c3a None admin] PATH_INFO: `/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/f18f635bff0142409e90da66547927f1/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:00.809440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd2270-198c-400e-82ba-6b5e2ad6e886 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:00.811694 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7f5fa-90fd-41cb-ac93-32b762ab5c3a None admin] RBAC: Authorizing `identity:create_grant(project_id=2fc6fd42a31942688f570f1ce86be510, user_id=f18f635bff0142409e90da66547927f1, role_id=c0cbd2d231594ea2bf7bb013fbf9ad1f)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:00.812271 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:00.814086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7f5fa-90fd-41cb-ac93-32b762ab5c3a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:00.836151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 301/610] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:41:00 2022] PUT /identity/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/f18f635bff0142409e90da66547927f1/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:41:00.856005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f517a-ec89-4a2d-a004-575eea24bbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:00.857007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f517a-ec89-4a2d-a004-575eea24bbd4 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:00.857597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f517a-ec89-4a2d-a004-575eea24bbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:00.858051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f517a-ec89-4a2d-a004-575eea24bbd4 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:00.859010 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f517a-ec89-4a2d-a004-575eea24bbd4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:00.859499 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f517a-ec89-4a2d-a004-575eea24bbd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:00.861178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f517a-ec89-4a2d-a004-575eea24bbd4 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:00.872280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 310/611] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:41:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:00.878945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7f5fa-90fd-41cb-ac93-32b762ab5c3a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:01.017389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7f5fa-90fd-41cb-ac93-32b762ab5c3a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:01.063556 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ed2cc-18ef-42ef-8b0e-89729df8f362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:01.064451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ed2cc-18ef-42ef-8b0e-89729df8f362 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:01.064829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ed2cc-18ef-42ef-8b0e-89729df8f362 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:01.065217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ed2cc-18ef-42ef-8b0e-89729df8f362 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:01.083375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ed2cc-18ef-42ef-8b0e-89729df8f362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:01.084978 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ed2cc-18ef-42ef-8b0e-89729df8f362 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:01.099062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-4d0ed2cc-18ef-42ef-8b0e-89729df8f362 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:01.112700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:01.114016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:01.115138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:01.115138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:01.115138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:01.115138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:01.116993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 302/612] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:01.124838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f517a-ec89-4a2d-a004-575eea24bbd4 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:01.243740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f517a-ec89-4a2d-a004-575eea24bbd4 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:01.287202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372467b-9911-484e-aab0-b62261c6a58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:01.287877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372467b-9911-484e-aab0-b62261c6a58e None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:01.288164 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372467b-9911-484e-aab0-b62261c6a58e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:01.288442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372467b-9911-484e-aab0-b62261c6a58e None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:01.289169 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372467b-9911-484e-aab0-b62261c6a58e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:01.289482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372467b-9911-484e-aab0-b62261c6a58e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:01.290838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372467b-9911-484e-aab0-b62261c6a58e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:01.308465 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 311/613] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:01.316401 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ed2cc-18ef-42ef-8b0e-89729df8f362 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:01.520631 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ed2cc-18ef-42ef-8b0e-89729df8f362 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:01.569698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309ded5-b891-4fcf-9f6c-ad72ae29adc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:01.570422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309ded5-b891-4fcf-9f6c-ad72ae29adc6 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:01.570711 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309ded5-b891-4fcf-9f6c-ad72ae29adc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:01.571022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309ded5-b891-4fcf-9f6c-ad72ae29adc6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:01.584983 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309ded5-b891-4fcf-9f6c-ad72ae29adc6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:01.586450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309ded5-b891-4fcf-9f6c-ad72ae29adc6 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-0309ded5-b891-4fcf-9f6c-ad72ae29adc6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:01.604498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:01.605479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:01.606654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:01.606654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:01.606654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:01.606654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:01.606654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:01.606654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:01.606654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:01.606654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:01.606654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 13:41:01.606654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:01.606654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 303/614] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:41:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:01.614671 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372467b-9911-484e-aab0-b62261c6a58e None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:01.665068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:01.750149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372467b-9911-484e-aab0-b62261c6a58e None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:01.787380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45cfd3-4bf4-4856-8ccc-8d57a1a3fae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:01.788056 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45cfd3-4bf4-4856-8ccc-8d57a1a3fae4 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:01.788341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45cfd3-4bf4-4856-8ccc-8d57a1a3fae4 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:01.788619 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45cfd3-4bf4-4856-8ccc-8d57a1a3fae4 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:01.789458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b45cfd3-4bf4-4856-8ccc-8d57a1a3fae4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:01.789770 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b45cfd3-4bf4-4856-8ccc-8d57a1a3fae4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:01.791224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b45cfd3-4bf4-4856-8ccc-8d57a1a3fae4 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:01.809684 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 312/615] 10.222.0.52 () {56 vars in 1135 bytes} [Thu May 19 13:41:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:01.815228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309ded5-b891-4fcf-9f6c-ad72ae29adc6 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:01.930430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a16033-cb12-4f07-adc5-44836a6cd798 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:01.930824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a16033-cb12-4f07-adc5-44836a6cd798 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:01.931051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a16033-cb12-4f07-adc5-44836a6cd798 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:01.932660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 313/616] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:01.937692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a0fd-e308-45f2-9a2d-c1ad0583d516 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:01.937983 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a0fd-e308-45f2-9a2d-c1ad0583d516 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:01.938262 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a0fd-e308-45f2-9a2d-c1ad0583d516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:01.942707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:01.955699 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-8354a0fd-e308-45f2-9a2d-c1ad0583d516 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:01.957129 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8354a0fd-e308-45f2-9a2d-c1ad0583d516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:01.995411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309ded5-b891-4fcf-9f6c-ad72ae29adc6 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:02.033303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 314/617] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:41:02.058270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354a0fd-e308-45f2-9a2d-c1ad0583d516 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:02.072967 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5dfb3-449c-4bbd-ad03-656abe7737eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:02.073769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5dfb3-449c-4bbd-ad03-656abe7737eb None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:02.074050 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5dfb3-449c-4bbd-ad03-656abe7737eb None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:02.074155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:02.074329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5dfb3-449c-4bbd-ad03-656abe7737eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:02.087107 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5dfb3-449c-4bbd-ad03-656abe7737eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:02.088534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5dfb3-449c-4bbd-ad03-656abe7737eb None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-62b5dfb3-449c-4bbd-ad03-656abe7737eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:02.120392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:02.121326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:02.122445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:02.122445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:02.122445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:02.122445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:02.131902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 304/618] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:02.139785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5dfb3-449c-4bbd-ad03-656abe7737eb None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:02.184057 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354a0fd-e308-45f2-9a2d-c1ad0583d516 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:02.221349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81f7d3-c4a4-410c-90b0-5f0394781b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:02.222253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81f7d3-c4a4-410c-90b0-5f0394781b08 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:02.222547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81f7d3-c4a4-410c-90b0-5f0394781b08 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:02.222820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81f7d3-c4a4-410c-90b0-5f0394781b08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:02.234121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81f7d3-c4a4-410c-90b0-5f0394781b08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-3d81f7d3-c4a4-410c-90b0-5f0394781b08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:02.242613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:02.243630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:02.245390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:02.245390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:02.245390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:02.245390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:02.245390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:02.245390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:02.245390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:02.245390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:02.245390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:02.245390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:02.245390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:02.245390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:02.245390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 315/619] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:41:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:02.254890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81f7d3-c4a4-410c-90b0-5f0394781b08 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:02.288468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5dfb3-449c-4bbd-ad03-656abe7737eb None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:02.317661 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:02.341151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:02.371901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81f7d3-c4a4-410c-90b0-5f0394781b08 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:02.375613 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff34730-b00b-493a-9cce-f8a4f822da94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:02.376599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff34730-b00b-493a-9cce-f8a4f822da94 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:02.377085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff34730-b00b-493a-9cce-f8a4f822da94 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:02.377577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff34730-b00b-493a-9cce-f8a4f822da94 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:02.379107 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff34730-b00b-493a-9cce-f8a4f822da94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:02.379619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff34730-b00b-493a-9cce-f8a4f822da94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:02.381198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff34730-b00b-493a-9cce-f8a4f822da94 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:02.397889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 305/620] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:02.399769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff34730-b00b-493a-9cce-f8a4f822da94 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:02.419795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a78716-cdef-448f-9bde-ca9b2095272c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:02.420600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a78716-cdef-448f-9bde-ca9b2095272c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:02.420886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a78716-cdef-448f-9bde-ca9b2095272c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:02.421158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a78716-cdef-448f-9bde-ca9b2095272c None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:02.421906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a78716-cdef-448f-9bde-ca9b2095272c None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:02.422322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a78716-cdef-448f-9bde-ca9b2095272c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:02.423779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a78716-cdef-448f-9bde-ca9b2095272c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:02.433236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 316/621] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:41:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:02.438201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a78716-cdef-448f-9bde-ca9b2095272c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:02.555835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a78716-cdef-448f-9bde-ca9b2095272c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:02.591904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cacd5-9790-4f3f-8fc7-fcce0be16aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:02.592735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cacd5-9790-4f3f-8fc7-fcce0be16aaf None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:02.593025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cacd5-9790-4f3f-8fc7-fcce0be16aaf None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:02.593303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cacd5-9790-4f3f-8fc7-fcce0be16aaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:02.606480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71cacd5-9790-4f3f-8fc7-fcce0be16aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:02.608185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71cacd5-9790-4f3f-8fc7-fcce0be16aaf None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:02.610363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff34730-b00b-493a-9cce-f8a4f822da94 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:02.611291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-e71cacd5-9790-4f3f-8fc7-fcce0be16aaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:02.643299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:02.645229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:02.646105 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:02.646105 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:02.646105 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:02.646105 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:02.648812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 317/622] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:02.651806 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cacd5-9790-4f3f-8fc7-fcce0be16aaf None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:02.673134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bf33a-5cae-4877-b7a7-0c41478440ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:02.674633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bf33a-5cae-4877-b7a7-0c41478440ca None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:02.675702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bf33a-5cae-4877-b7a7-0c41478440ca None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:02.675983 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bf33a-5cae-4877-b7a7-0c41478440ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:02.687342 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bf33a-5cae-4877-b7a7-0c41478440ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:02.689082 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bf33a-5cae-4877-b7a7-0c41478440ca None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-3f0bf33a-5cae-4877-b7a7-0c41478440ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:02.713155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:02.715552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:02.717775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:02.717775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:02.717775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:02.717775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:02.717775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:02.717775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:02.717775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:02.717775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:02.717775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 306/623] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:02.723915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bf33a-5cae-4877-b7a7-0c41478440ca None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:02.807722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cacd5-9790-4f3f-8fc7-fcce0be16aaf None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:02.816118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:02.850051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658e8e1-4096-48b0-a1d3-88db69f79df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:02.851152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658e8e1-4096-48b0-a1d3-88db69f79df0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:02.851667 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658e8e1-4096-48b0-a1d3-88db69f79df0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:02.852873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658e8e1-4096-48b0-a1d3-88db69f79df0 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:02.853895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658e8e1-4096-48b0-a1d3-88db69f79df0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:02.856183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658e8e1-4096-48b0-a1d3-88db69f79df0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:02.857799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658e8e1-4096-48b0-a1d3-88db69f79df0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:02.884795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 318/624] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:02.894365 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658e8e1-4096-48b0-a1d3-88db69f79df0 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:02.956172 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bf33a-5cae-4877-b7a7-0c41478440ca None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:03.001752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:03.041469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4b8ef-9c4d-4866-851d-95bfc0d5139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:03.042140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4b8ef-9c4d-4866-851d-95bfc0d5139f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:03.042431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4b8ef-9c4d-4866-851d-95bfc0d5139f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:03.042709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4b8ef-9c4d-4866-851d-95bfc0d5139f None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:03.043526 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4b8ef-9c4d-4866-851d-95bfc0d5139f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:03.043835 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4b8ef-9c4d-4866-851d-95bfc0d5139f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:03.045259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4b8ef-9c4d-4866-851d-95bfc0d5139f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:03.073046 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 307/625] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:03.076781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4b8ef-9c4d-4866-851d-95bfc0d5139f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:03.119419 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658e8e1-4096-48b0-a1d3-88db69f79df0 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:03.196349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109219fc-ee50-4a0b-8da0-ad8496c78459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:03.197411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109219fc-ee50-4a0b-8da0-ad8496c78459 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:03.197525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109219fc-ee50-4a0b-8da0-ad8496c78459 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:03.197760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109219fc-ee50-4a0b-8da0-ad8496c78459 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:03.205908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109219fc-ee50-4a0b-8da0-ad8496c78459 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:03.208375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109219fc-ee50-4a0b-8da0-ad8496c78459 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-109219fc-ee50-4a0b-8da0-ad8496c78459 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:03.225705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:03.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:03.227755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:03.227755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:03.227755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:03.227755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:03.227755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:03.227755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:03.227755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:03.227755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:03.227755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:41:03.227755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:03.227755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 319/626] 10.222.0.52 () {56 vars in 1093 bytes} [Thu May 19 13:41:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:03.232406 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89d3e2-0538-4932-9013-a9f4ac58eee1 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:03.232698 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89d3e2-0538-4932-9013-a9f4ac58eee1 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:03.232978 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89d3e2-0538-4932-9013-a9f4ac58eee1 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:03.235418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 320/627] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:03.236605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89d3e2-0538-4932-9013-a9f4ac58eee1 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:03.247816 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:03.258631 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4b8ef-9c4d-4866-851d-95bfc0d5139f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:03.300476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccd874-1f1d-4d06-aea3-1335cd72ba4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:03.301936 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccd874-1f1d-4d06-aea3-1335cd72ba4a None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:03.301936 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccd874-1f1d-4d06-aea3-1335cd72ba4a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:03.302481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccd874-1f1d-4d06-aea3-1335cd72ba4a None admin] PATH_INFO: `/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/8554ff40bbcb4fbb801d854d6527a6d2/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:03.318585 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccd874-1f1d-4d06-aea3-1335cd72ba4a None admin] RBAC: Authorizing `identity:create_grant(project_id=2fc6fd42a31942688f570f1ce86be510, user_id=8554ff40bbcb4fbb801d854d6527a6d2, role_id=c0cbd2d231594ea2bf7bb013fbf9ad1f)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:03.320087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccd874-1f1d-4d06-aea3-1335cd72ba4a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:03.334491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 308/628] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:41:03 2022] PUT /identity/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/8554ff40bbcb4fbb801d854d6527a6d2/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:41:03.338278 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959bb37-4c9f-4015-89ad-e67dffed4e36 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:03.339306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959bb37-4c9f-4015-89ad-e67dffed4e36 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:03.341537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959bb37-4c9f-4015-89ad-e67dffed4e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:03.369468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-1959bb37-4c9f-4015-89ad-e67dffed4e36 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:03.372535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1959bb37-4c9f-4015-89ad-e67dffed4e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:03.381410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89d3e2-0538-4932-9013-a9f4ac58eee1 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:03.406566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:03.442767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f077c8f-ad16-4c97-8730-54ae64831068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:03.443517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f077c8f-ad16-4c97-8730-54ae64831068 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:03.443801 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f077c8f-ad16-4c97-8730-54ae64831068 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:03.444077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f077c8f-ad16-4c97-8730-54ae64831068 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:03.445147 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f077c8f-ad16-4c97-8730-54ae64831068 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:03.445415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f077c8f-ad16-4c97-8730-54ae64831068 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:03.447463 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f077c8f-ad16-4c97-8730-54ae64831068 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:03.475394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 321/629] 10.222.0.52 () {56 vars in 1133 bytes} [Thu May 19 13:41:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:03.476959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa08797-7e22-47d3-a3ff-a47a07503479 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:03.477252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa08797-7e22-47d3-a3ff-a47a07503479 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:03.477530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa08797-7e22-47d3-a3ff-a47a07503479 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:03.479362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 322/630] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:03.480119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa08797-7e22-47d3-a3ff-a47a07503479 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:03.487181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 309/631] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:41:03.489101 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1a1ba-548b-41eb-88d9-f544e00466ec None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:03.489688 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1a1ba-548b-41eb-88d9-f544e00466ec None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:03.490166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1a1ba-548b-41eb-88d9-f544e00466ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:03.509646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-8ea1a1ba-548b-41eb-88d9-f544e00466ec None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:03.511314 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-8ea1a1ba-548b-41eb-88d9-f544e00466ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:03.591507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 310/632] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:41:03.592828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1a1ba-548b-41eb-88d9-f544e00466ec None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:03.595910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:03.623087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa08797-7e22-47d3-a3ff-a47a07503479 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:03.660689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82c559-64a2-4e9d-8a79-e6d44deabe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:03.661459 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82c559-64a2-4e9d-8a79-e6d44deabe40 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:03.661737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82c559-64a2-4e9d-8a79-e6d44deabe40 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:03.662014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82c559-64a2-4e9d-8a79-e6d44deabe40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:03.701665 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:03.703776 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1a1ba-548b-41eb-88d9-f544e00466ec None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:03.739831 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f489667-52bb-451f-bc59-ac267b6f8819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:03.740549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f489667-52bb-451f-bc59-ac267b6f8819 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:03.740829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f489667-52bb-451f-bc59-ac267b6f8819 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:03.741217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f489667-52bb-451f-bc59-ac267b6f8819 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:03.742202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82c559-64a2-4e9d-8a79-e6d44deabe40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:03.743757 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82c559-64a2-4e9d-8a79-e6d44deabe40 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:03.752059 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f489667-52bb-451f-bc59-ac267b6f8819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-5f489667-52bb-451f-bc59-ac267b6f8819 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:03.760357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:03.761498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:03.762620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:03.762620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:03.762620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:03.762620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:03.762620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:03.762620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:03.762620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:03.762620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:03.762620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:03.762620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:03.762620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:03.762620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:03.762620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 311/633] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:41:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:03.763366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2725c7-0210-421d-8aa5-de6dc82fbe55 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:03.763366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2725c7-0210-421d-8aa5-de6dc82fbe55 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:03.763637 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2725c7-0210-421d-8aa5-de6dc82fbe55 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:03.767245 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 312/634] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:03.767245 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2725c7-0210-421d-8aa5-de6dc82fbe55 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:03.835740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:03.863656 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 323/635] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:41:03.864585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82c559-64a2-4e9d-8a79-e6d44deabe40 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:03.883608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2725c7-0210-421d-8aa5-de6dc82fbe55 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:03.917799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253b37a-f9b5-4a94-924c-188c1e45de75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:03.918594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253b37a-f9b5-4a94-924c-188c1e45de75 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:03.918594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253b37a-f9b5-4a94-924c-188c1e45de75 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:03.918594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253b37a-f9b5-4a94-924c-188c1e45de75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:03.930102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253b37a-f9b5-4a94-924c-188c1e45de75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:03.934850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-4253b37a-f9b5-4a94-924c-188c1e45de75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:03.937949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:03.939462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:03.940317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:03.940317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:03.940317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:03.940317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:03.940317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:03.940317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:03.940317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:03.940317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:03.940317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:03.940317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:03.940317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:03.940317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:03.940317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 313/636] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:41:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:03.941310 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622b4bd-c074-4a6c-b18f-9684b572aa8b None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:03.941310 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622b4bd-c074-4a6c-b18f-9684b572aa8b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:03.941530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622b4bd-c074-4a6c-b18f-9684b572aa8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:03.958607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-2622b4bd-c074-4a6c-b18f-9684b572aa8b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:03.959848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2622b4bd-c074-4a6c-b18f-9684b572aa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:03.980306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82c559-64a2-4e9d-8a79-e6d44deabe40 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:04.018427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbeced3-cd5a-433e-8a6c-cd871286778a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:04.019702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbeced3-cd5a-433e-8a6c-cd871286778a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:04.020000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbeced3-cd5a-433e-8a6c-cd871286778a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:04.020275 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbeced3-cd5a-433e-8a6c-cd871286778a None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:04.021030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbeced3-cd5a-433e-8a6c-cd871286778a None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:04.021341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbeced3-cd5a-433e-8a6c-cd871286778a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:04.023014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbeced3-cd5a-433e-8a6c-cd871286778a None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:04.031330 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 314/637] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:41:04.031330 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622b4bd-c074-4a6c-b18f-9684b572aa8b None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:04.036351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 324/638] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:41:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:04.037201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbeced3-cd5a-433e-8a6c-cd871286778a None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:04.093407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:04.145370 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622b4bd-c074-4a6c-b18f-9684b572aa8b None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:04.151160 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbeced3-cd5a-433e-8a6c-cd871286778a None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:04.179206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:04.184030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefb0e1-5b91-4a62-af40-cb9af0ce675a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:04.184852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefb0e1-5b91-4a62-af40-cb9af0ce675a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:04.185138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefb0e1-5b91-4a62-af40-cb9af0ce675a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:04.185441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefb0e1-5b91-4a62-af40-cb9af0ce675a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:04.189068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d48b9-b115-4645-8e08-fc780e3f8f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:04.189994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d48b9-b115-4645-8e08-fc780e3f8f71 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:04.190685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d48b9-b115-4645-8e08-fc780e3f8f71 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:04.191278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d48b9-b115-4645-8e08-fc780e3f8f71 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:04.192226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d48b9-b115-4645-8e08-fc780e3f8f71 None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:04.192745 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d48b9-b115-4645-8e08-fc780e3f8f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:04.194272 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d48b9-b115-4645-8e08-fc780e3f8f71 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:04.196525 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daefb0e1-5b91-4a62-af40-cb9af0ce675a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:04.197962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daefb0e1-5b91-4a62-af40-cb9af0ce675a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:04.203575 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 325/639] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:41:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:04.204673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d48b9-b115-4645-8e08-fc780e3f8f71 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-daefb0e1-5b91-4a62-af40-cb9af0ce675a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:04.219345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:04.220482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:04.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:04.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:04.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:04.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:04.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 315/640] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:04.226768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefb0e1-5b91-4a62-af40-cb9af0ce675a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:04.317606 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d48b9-b115-4645-8e08-fc780e3f8f71 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:04.329021 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefb0e1-5b91-4a62-af40-cb9af0ce675a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:04.332667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:04.355032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e6700-043a-4a99-abf0-467968d3b439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:04.355823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e6700-043a-4a99-abf0-467968d3b439 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:04.356108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e6700-043a-4a99-abf0-467968d3b439 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:04.356374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e6700-043a-4a99-abf0-467968d3b439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:04.367929 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e6700-043a-4a99-abf0-467968d3b439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:04.368862 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1999d-e5d2-45e7-98b3-64f16a73b418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:04.369374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e6700-043a-4a99-abf0-467968d3b439 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:04.369656 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1999d-e5d2-45e7-98b3-64f16a73b418 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:04.369941 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1999d-e5d2-45e7-98b3-64f16a73b418 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:04.370216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1999d-e5d2-45e7-98b3-64f16a73b418 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:04.381392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1999d-e5d2-45e7-98b3-64f16a73b418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-dbf1999d-e5d2-45e7-98b3-64f16a73b418 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:04.389036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:04.389959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:04.390798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:04.390798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:04.390798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:04.390798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:04.390798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:04.390798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:04.390798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:04.390798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:04.390798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:04.390798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:04.390798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:04.390798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:04.390798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 316/641] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:41:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:04.392021 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1999d-e5d2-45e7-98b3-64f16a73b418 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-cd8e6700-043a-4a99-abf0-467968d3b439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:04.397746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:04.398689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:04.399654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:04.399654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:04.399654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:04.399654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:04.402295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 326/642] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:04.403675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e6700-043a-4a99-abf0-467968d3b439 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:04.451265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:04.504783 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1999d-e5d2-45e7-98b3-64f16a73b418 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:04.513240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e6700-043a-4a99-abf0-467968d3b439 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:04.538497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e281f3d-3b3f-4504-82f2-48aa04800e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:04.539323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e281f3d-3b3f-4504-82f2-48aa04800e9c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:04.539609 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e281f3d-3b3f-4504-82f2-48aa04800e9c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:04.539883 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e281f3d-3b3f-4504-82f2-48aa04800e9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:04.551420 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e281f3d-3b3f-4504-82f2-48aa04800e9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:04.551860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd14f9-8591-46ea-9384-39b677e5c7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:04.552659 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd14f9-8591-46ea-9384-39b677e5c7c4 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:04.552883 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e281f3d-3b3f-4504-82f2-48aa04800e9c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:04.553145 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd14f9-8591-46ea-9384-39b677e5c7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:04.553720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd14f9-8591-46ea-9384-39b677e5c7c4 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:04.554593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd14f9-8591-46ea-9384-39b677e5c7c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:04.555048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd14f9-8591-46ea-9384-39b677e5c7c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:04.556570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd14f9-8591-46ea-9384-39b677e5c7c4 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-7e281f3d-3b3f-4504-82f2-48aa04800e9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:04.571841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:04.572750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:04.573712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:04.573712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:04.573712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:04.573712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:04.574702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 327/643] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:04.575034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 317/644] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:04.575658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd14f9-8591-46ea-9384-39b677e5c7c4 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:04.576414 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e281f3d-3b3f-4504-82f2-48aa04800e9c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:04.598320 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:04.684919 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd14f9-8591-46ea-9384-39b677e5c7c4 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:04.686651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:04.688124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e281f3d-3b3f-4504-82f2-48aa04800e9c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:04.723648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93e9cc-8eb6-42fa-b8a5-231698805829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:04.724319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93e9cc-8eb6-42fa-b8a5-231698805829 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:04.724604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93e9cc-8eb6-42fa-b8a5-231698805829 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:04.724718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12631df7-c15b-4a89-8311-55d05cf77e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:04.725138 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93e9cc-8eb6-42fa-b8a5-231698805829 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:04.725400 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12631df7-c15b-4a89-8311-55d05cf77e4e None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:04.725614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93e9cc-8eb6-42fa-b8a5-231698805829 None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:04.725723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12631df7-c15b-4a89-8311-55d05cf77e4e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:04.725927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93e9cc-8eb6-42fa-b8a5-231698805829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:04.726024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12631df7-c15b-4a89-8311-55d05cf77e4e None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:04.726646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12631df7-c15b-4a89-8311-55d05cf77e4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:04.726984 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12631df7-c15b-4a89-8311-55d05cf77e4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:04.727336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93e9cc-8eb6-42fa-b8a5-231698805829 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:04.728337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12631df7-c15b-4a89-8311-55d05cf77e4e None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:04.741177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 328/645] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:41:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:04.741177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93e9cc-8eb6-42fa-b8a5-231698805829 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:04.749334 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 318/646] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:04.749334 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12631df7-c15b-4a89-8311-55d05cf77e4e None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:04.840353 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:04.843106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93e9cc-8eb6-42fa-b8a5-231698805829 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:04.867692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12631df7-c15b-4a89-8311-55d05cf77e4e None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:04.880709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dbdef-2872-44fa-b52b-ae65db44ded0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:04.881575 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dbdef-2872-44fa-b52b-ae65db44ded0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:04.881869 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dbdef-2872-44fa-b52b-ae65db44ded0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:04.882153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dbdef-2872-44fa-b52b-ae65db44ded0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:04.889171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dbdef-2872-44fa-b52b-ae65db44ded0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:04.890620 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dbdef-2872-44fa-b52b-ae65db44ded0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:04.904409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd7c9a-10ab-4e5f-a40e-e5b7eb9d99a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-507dbdef-2872-44fa-b52b-ae65db44ded0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:04.905035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:04.905879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:04.907391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:04.907391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:04.907391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:04.907391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:04.907391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:04.907391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:04.907391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:04.907391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:04.907829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd7c9a-10ab-4e5f-a40e-e5b7eb9d99a8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:04.907829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd7c9a-10ab-4e5f-a40e-e5b7eb9d99a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:04.907829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd7c9a-10ab-4e5f-a40e-e5b7eb9d99a8 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:04.908008 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 329/647] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:04.908008 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dbdef-2872-44fa-b52b-ae65db44ded0 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:04.911596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd7c9a-10ab-4e5f-a40e-e5b7eb9d99a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:04.911973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd7c9a-10ab-4e5f-a40e-e5b7eb9d99a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:04.914199 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd7c9a-10ab-4e5f-a40e-e5b7eb9d99a8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:04.929253 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 319/648] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:04.933770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd7c9a-10ab-4e5f-a40e-e5b7eb9d99a8 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:04.934385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:05.014028 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dbdef-2872-44fa-b52b-ae65db44ded0 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:05.047443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd7c9a-10ab-4e5f-a40e-e5b7eb9d99a8 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:05.048944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec35430-5e31-42aa-9e25-63fa1e5c53c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:05.051329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec35430-5e31-42aa-9e25-63fa1e5c53c2 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:05.051835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec35430-5e31-42aa-9e25-63fa1e5c53c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:05.052300 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec35430-5e31-42aa-9e25-63fa1e5c53c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:05.064360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec35430-5e31-42aa-9e25-63fa1e5c53c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:05.066095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec35430-5e31-42aa-9e25-63fa1e5c53c2 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:05.081955 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735ac4a-cc25-43d7-b425-3ccde2c8a4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:05.082755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735ac4a-cc25-43d7-b425-3ccde2c8a4f0 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:05.083065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735ac4a-cc25-43d7-b425-3ccde2c8a4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:05.083342 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735ac4a-cc25-43d7-b425-3ccde2c8a4f0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-6ec35430-5e31-42aa-9e25-63fa1e5c53c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:05.085212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:05.086291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:05.087255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:05.087255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:05.087255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:05.087255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:05.087255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 330/649] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:05.087629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec35430-5e31-42aa-9e25-63fa1e5c53c2 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:05.088664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:05.098271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735ac4a-cc25-43d7-b425-3ccde2c8a4f0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:05.099646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735ac4a-cc25-43d7-b425-3ccde2c8a4f0 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-9735ac4a-cc25-43d7-b425-3ccde2c8a4f0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:05.114974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:05.115949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:05.116920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:05.116920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:05.116920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:05.116920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:05.116920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:05.116920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:05.116920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:05.116920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:05.116920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:41:05.116920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:05.116920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 320/650] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:41:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:05.117515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735ac4a-cc25-43d7-b425-3ccde2c8a4f0 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:05.175766 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:05.199579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec35430-5e31-42aa-9e25-63fa1e5c53c2 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:05.229987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735ac4a-cc25-43d7-b425-3ccde2c8a4f0 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:05.237018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e46ef-9f15-417a-b798-d02c5155fdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:05.238008 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e46ef-9f15-417a-b798-d02c5155fdfe None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:05.238504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e46ef-9f15-417a-b798-d02c5155fdfe None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:05.239155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e46ef-9f15-417a-b798-d02c5155fdfe None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:05.240152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e46ef-9f15-417a-b798-d02c5155fdfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:05.240666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e46ef-9f15-417a-b798-d02c5155fdfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:05.242297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e46ef-9f15-417a-b798-d02c5155fdfe None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:05.256537 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 331/651] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:05.257643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e46ef-9f15-417a-b798-d02c5155fdfe None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:05.269993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6965558-40f2-4f14-bc77-650f18e88779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:05.270784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6965558-40f2-4f14-bc77-650f18e88779 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:05.271095 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6965558-40f2-4f14-bc77-650f18e88779 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:05.271370 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6965558-40f2-4f14-bc77-650f18e88779 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:05.279312 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6965558-40f2-4f14-bc77-650f18e88779 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:05.280844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6965558-40f2-4f14-bc77-650f18e88779 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-b6965558-40f2-4f14-bc77-650f18e88779 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:05.296643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:05.297616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:05.298589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:05.298589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:05.298589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:05.298589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:05.298589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:05.298589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:05.298589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:05.298589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:05.298589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 13:41:05.298589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:05.298589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 321/652] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:41:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:05.299425 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6965558-40f2-4f14-bc77-650f18e88779 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:05.360799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:05.390276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e46ef-9f15-417a-b798-d02c5155fdfe None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:05.417170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6965558-40f2-4f14-bc77-650f18e88779 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:05.429340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a352b6-daa9-44b6-8937-3a35e0b9d5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:05.430014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a352b6-daa9-44b6-8937-3a35e0b9d5e9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:05.430302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a352b6-daa9-44b6-8937-3a35e0b9d5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:05.430582 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a352b6-daa9-44b6-8937-3a35e0b9d5e9 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:05.431339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a352b6-daa9-44b6-8937-3a35e0b9d5e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:05.431650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a352b6-daa9-44b6-8937-3a35e0b9d5e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:05.433020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a352b6-daa9-44b6-8937-3a35e0b9d5e9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:05.435886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:05.451308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 332/653] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:05.451308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a352b6-daa9-44b6-8937-3a35e0b9d5e9 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:05.458384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69677b5f-54ba-4c2a-86d0-e311d42c4d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:05.459099 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69677b5f-54ba-4c2a-86d0-e311d42c4d93 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:05.459741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69677b5f-54ba-4c2a-86d0-e311d42c4d93 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:05.460026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69677b5f-54ba-4c2a-86d0-e311d42c4d93 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:05.461183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69677b5f-54ba-4c2a-86d0-e311d42c4d93 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:05.461499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69677b5f-54ba-4c2a-86d0-e311d42c4d93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:05.463101 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69677b5f-54ba-4c2a-86d0-e311d42c4d93 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:05.486906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 322/654] 10.222.0.52 () {56 vars in 1137 bytes} [Thu May 19 13:41:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:05.486906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69677b5f-54ba-4c2a-86d0-e311d42c4d93 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:05.568151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a352b6-daa9-44b6-8937-3a35e0b9d5e9 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:05.598086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69677b5f-54ba-4c2a-86d0-e311d42c4d93 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:05.604647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b0917-3e9a-4c99-861c-cf92f8ef4d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:05.605308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b0917-3e9a-4c99-861c-cf92f8ef4d39 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:05.605591 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b0917-3e9a-4c99-861c-cf92f8ef4d39 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:05.606135 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b0917-3e9a-4c99-861c-cf92f8ef4d39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:05.606807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b0917-3e9a-4c99-861c-cf92f8ef4d39 None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=4f6093d0c1fc41d98c58bace81e3d8e5) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:05.607147 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b0917-3e9a-4c99-861c-cf92f8ef4d39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:05.608672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b0917-3e9a-4c99-861c-cf92f8ef4d39 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:05.614622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:05.615747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 333/655] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:05 2022] GET /identity/v3/role_assignments?user.id=4f6093d0c1fc41d98c58bace81e3d8e5&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:05.616634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b0917-3e9a-4c99-861c-cf92f8ef4d39 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:05.657761 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293701d8-d4fc-4d8c-b391-18dcb5d60a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:05.658961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293701d8-d4fc-4d8c-b391-18dcb5d60a98 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:05.659998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293701d8-d4fc-4d8c-b391-18dcb5d60a98 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:05.660736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293701d8-d4fc-4d8c-b391-18dcb5d60a98 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:05.661881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293701d8-d4fc-4d8c-b391-18dcb5d60a98 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:05.662406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293701d8-d4fc-4d8c-b391-18dcb5d60a98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:05.664859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293701d8-d4fc-4d8c-b391-18dcb5d60a98 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:05.687229 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 323/656] 10.222.0.52 () {56 vars in 1137 bytes} [Thu May 19 13:41:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:05.688108 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293701d8-d4fc-4d8c-b391-18dcb5d60a98 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:05.696582 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:05.747867 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b0917-3e9a-4c99-861c-cf92f8ef4d39 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:05.785877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e2560-e38f-417a-ab23-bcfae52858d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:05.786679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e2560-e38f-417a-ab23-bcfae52858d8 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:05.786986 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e2560-e38f-417a-ab23-bcfae52858d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:05.787258 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e2560-e38f-417a-ab23-bcfae52858d8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:05.801119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e2560-e38f-417a-ab23-bcfae52858d8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:05.802604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e2560-e38f-417a-ab23-bcfae52858d8 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:05.805329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293701d8-d4fc-4d8c-b391-18dcb5d60a98 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-9b7e2560-e38f-417a-ab23-bcfae52858d8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:05.818708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:05.819733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:05.820660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:05.820660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:05.820660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:05.820660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:05.820660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:05.820660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:05.820660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:05.820660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:05.820660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 13:41:05.820660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:05.820660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 334/657] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:41:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:05.821249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c64c93-b867-4186-9ddb-1a8f8b5a2f4c None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:05.821933 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c64c93-b867-4186-9ddb-1a8f8b5a2f4c None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:05.821933 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c64c93-b867-4186-9ddb-1a8f8b5a2f4c None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:05.824399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 335/658] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:05.824399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c64c93-b867-4186-9ddb-1a8f8b5a2f4c None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:05.853896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad650155-be0a-40ef-af34-20eca85140f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:05.854558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad650155-be0a-40ef-af34-20eca85140f6 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:05.854841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad650155-be0a-40ef-af34-20eca85140f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:05.855147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad650155-be0a-40ef-af34-20eca85140f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:05.904308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:05.932831 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad650155-be0a-40ef-af34-20eca85140f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:05.934280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad650155-be0a-40ef-af34-20eca85140f6 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:05.949867 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c64c93-b867-4186-9ddb-1a8f8b5a2f4c None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:05.973684 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:05.989483 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcc332-f5d2-46a4-a6da-7d2c3d4d65b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:05.990257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcc332-f5d2-46a4-a6da-7d2c3d4d65b1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:05.990545 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcc332-f5d2-46a4-a6da-7d2c3d4d65b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:05.990819 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcc332-f5d2-46a4-a6da-7d2c3d4d65b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:06.035614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 324/659] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:41:06.036290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad650155-be0a-40ef-af34-20eca85140f6 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:06.069356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcc332-f5d2-46a4-a6da-7d2c3d4d65b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:06.070832 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcc332-f5d2-46a4-a6da-7d2c3d4d65b1 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:06.109156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:06.139468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad650155-be0a-40ef-af34-20eca85140f6 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:06.178257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 336/660] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:41:06.178257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd0272-f711-4349-89fb-287b1e0b9259 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:06.178257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd0272-f711-4349-89fb-287b1e0b9259 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:06.178257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd0272-f711-4349-89fb-287b1e0b9259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:06.179533 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87b52f-fcd7-4b37-bc4f-4dcd9b4d8071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:06.180207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87b52f-fcd7-4b37-bc4f-4dcd9b4d8071 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:06.180494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87b52f-fcd7-4b37-bc4f-4dcd9b4d8071 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:06.180773 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87b52f-fcd7-4b37-bc4f-4dcd9b4d8071 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:06.181612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87b52f-fcd7-4b37-bc4f-4dcd9b4d8071 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:06.181923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87b52f-fcd7-4b37-bc4f-4dcd9b4d8071 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:06.183381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87b52f-fcd7-4b37-bc4f-4dcd9b4d8071 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:06.194091 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-86dd0272-f711-4349-89fb-287b1e0b9259 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:06.195563 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-86dd0272-f711-4349-89fb-287b1e0b9259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:06.205575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 325/661] 10.222.0.52 () {56 vars in 1139 bytes} [Thu May 19 13:41:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:06.205575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87b52f-fcd7-4b37-bc4f-4dcd9b4d8071 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:06.216622 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:06.277567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 337/662] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:41:06.278748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd0272-f711-4349-89fb-287b1e0b9259 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:06.322248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87b52f-fcd7-4b37-bc4f-4dcd9b4d8071 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:06.355097 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:06.363907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c3f2a-98ba-457c-8877-16422de55a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:06.364715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3f2a-98ba-457c-8877-16422de55a2d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:06.365118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3f2a-98ba-457c-8877-16422de55a2d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:06.365397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3f2a-98ba-457c-8877-16422de55a2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:06.376626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c3f2a-98ba-457c-8877-16422de55a2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:06.378221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c3f2a-98ba-457c-8877-16422de55a2d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:06.392361 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd0272-f711-4349-89fb-287b1e0b9259 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-e07c3f2a-98ba-457c-8877-16422de55a2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:06.400142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:06.401001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:06.401875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:06.401875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:06.401875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:06.401875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:06.401875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 326/663] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:06.402623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c3f2a-98ba-457c-8877-16422de55a2d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:06.436728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4acdc1-7a62-4d56-9632-7526152f1990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:06.437442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4acdc1-7a62-4d56-9632-7526152f1990 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:06.437726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4acdc1-7a62-4d56-9632-7526152f1990 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:06.438213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4acdc1-7a62-4d56-9632-7526152f1990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:06.449223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4acdc1-7a62-4d56-9632-7526152f1990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:06.450676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4acdc1-7a62-4d56-9632-7526152f1990 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-de4acdc1-7a62-4d56-9632-7526152f1990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:06.470226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:06.471266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:06.472668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:06.472668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:06.472668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:06.472668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:06.472668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 338/664] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:06.472938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4acdc1-7a62-4d56-9632-7526152f1990 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:06.475915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:06.521453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c3f2a-98ba-457c-8877-16422de55a2d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:06.558810 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b3193-1450-43cd-95c7-98352e039aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:06.559788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b3193-1450-43cd-95c7-98352e039aab None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:06.560141 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b3193-1450-43cd-95c7-98352e039aab None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:06.560410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b3193-1450-43cd-95c7-98352e039aab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:06.589098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4acdc1-7a62-4d56-9632-7526152f1990 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:06.612020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:06.626448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605fc02-6291-48db-95e7-ca7a99cfc936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:06.627372 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605fc02-6291-48db-95e7-ca7a99cfc936 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:06.627654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605fc02-6291-48db-95e7-ca7a99cfc936 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:06.627934 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605fc02-6291-48db-95e7-ca7a99cfc936 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:06.633756 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b3193-1450-43cd-95c7-98352e039aab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:06.635228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b3193-1450-43cd-95c7-98352e039aab None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:06.639111 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605fc02-6291-48db-95e7-ca7a99cfc936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-9605fc02-6291-48db-95e7-ca7a99cfc936 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:06.647350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:06.648491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:06.649349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:06.649349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:06.649349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:06.649349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:06.649349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:06.649349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:06.649349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:06.649349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:06.649349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:06.649349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:06.649349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:06.649349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:06.649349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 339/665] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:41:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:06.650042 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605fc02-6291-48db-95e7-ca7a99cfc936 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:06.714945 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:06.749143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 327/666] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:41:06.749991 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b3193-1450-43cd-95c7-98352e039aab None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:06.762523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605fc02-6291-48db-95e7-ca7a99cfc936 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:06.797005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8bbe1-7046-46ea-ab47-a5f32ff9fd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:06.797667 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8bbe1-7046-46ea-ab47-a5f32ff9fd7d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:06.797959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8bbe1-7046-46ea-ab47-a5f32ff9fd7d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:06.798251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8bbe1-7046-46ea-ab47-a5f32ff9fd7d None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:06.798992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8bbe1-7046-46ea-ab47-a5f32ff9fd7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:06.799320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8bbe1-7046-46ea-ab47-a5f32ff9fd7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:06.800869 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8bbe1-7046-46ea-ab47-a5f32ff9fd7d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:06.817032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 340/667] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:06.817032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8bbe1-7046-46ea-ab47-a5f32ff9fd7d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:06.845148 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:06.864624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b3193-1450-43cd-95c7-98352e039aab None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:06.900068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de4f5f-b272-44c2-a0ec-0014bb4fb143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:06.900732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de4f5f-b272-44c2-a0ec-0014bb4fb143 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:06.901014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de4f5f-b272-44c2-a0ec-0014bb4fb143 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:06.901391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de4f5f-b272-44c2-a0ec-0014bb4fb143 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:06.901992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de4f5f-b272-44c2-a0ec-0014bb4fb143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:06.902298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de4f5f-b272-44c2-a0ec-0014bb4fb143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:06.903667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de4f5f-b272-44c2-a0ec-0014bb4fb143 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:06.916683 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 328/668] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:06.917494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de4f5f-b272-44c2-a0ec-0014bb4fb143 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:06.930991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8bbe1-7046-46ea-ab47-a5f32ff9fd7d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:06.956913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:06.970487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496cc82-e395-4d5c-b677-89f9334d9539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:06.971123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496cc82-e395-4d5c-b677-89f9334d9539 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:06.971403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496cc82-e395-4d5c-b677-89f9334d9539 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:06.971675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496cc82-e395-4d5c-b677-89f9334d9539 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:06.973072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496cc82-e395-4d5c-b677-89f9334d9539 None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:06.973541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496cc82-e395-4d5c-b677-89f9334d9539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:06.974932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496cc82-e395-4d5c-b677-89f9334d9539 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:06.987011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 341/669] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:41:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:06.987011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496cc82-e395-4d5c-b677-89f9334d9539 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:07.025629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de4f5f-b272-44c2-a0ec-0014bb4fb143 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:07.062028 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183615e9-9434-4136-a965-ac9d6f15fb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:07.062817 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183615e9-9434-4136-a965-ac9d6f15fb75 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:07.063132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183615e9-9434-4136-a965-ac9d6f15fb75 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:07.063415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183615e9-9434-4136-a965-ac9d6f15fb75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:07.075346 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183615e9-9434-4136-a965-ac9d6f15fb75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:07.076791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183615e9-9434-4136-a965-ac9d6f15fb75 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:07.078735 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:07.092839 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496cc82-e395-4d5c-b677-89f9334d9539 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-183615e9-9434-4136-a965-ac9d6f15fb75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:07.097203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:07.098326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:07.099840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:07.099840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:07.099840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:07.099840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:07.103647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 329/670] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:07.103937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183615e9-9434-4136-a965-ac9d6f15fb75 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:07.131264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab9562-c83e-4abc-b031-99c83113c031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:07.132045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab9562-c83e-4abc-b031-99c83113c031 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:07.132330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab9562-c83e-4abc-b031-99c83113c031 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:07.132600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab9562-c83e-4abc-b031-99c83113c031 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:07.139286 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab9562-c83e-4abc-b031-99c83113c031 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:07.140996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab9562-c83e-4abc-b031-99c83113c031 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-fbab9562-c83e-4abc-b031-99c83113c031 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:07.155500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:07.156385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:07.157280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:07.157280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:07.157280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:07.157280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:07.157280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:07.157280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:07.157280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:07.157280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:07.157280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 342/671] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:07.158432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab9562-c83e-4abc-b031-99c83113c031 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:07.205008 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:07.213337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183615e9-9434-4136-a965-ac9d6f15fb75 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:07.249291 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88d943-053d-41e3-9e40-b9c098cecaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:07.250018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88d943-053d-41e3-9e40-b9c098cecaf8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:07.250308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88d943-053d-41e3-9e40-b9c098cecaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:07.250585 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88d943-053d-41e3-9e40-b9c098cecaf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:07.257630 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88d943-053d-41e3-9e40-b9c098cecaf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:07.259090 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88d943-053d-41e3-9e40-b9c098cecaf8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:07.263022 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab9562-c83e-4abc-b031-99c83113c031 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-eb88d943-053d-41e3-9e40-b9c098cecaf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:07.273399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:07.274379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:07.275356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:07.275356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:07.275356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:07.275356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:07.275356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:07.275356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:07.275356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:07.275356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:07.279143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 330/672] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:07.279927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88d943-053d-41e3-9e40-b9c098cecaf8 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:07.303568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16a052-10aa-4fa8-afd1-292be2ba0a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:07.304682 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16a052-10aa-4fa8-afd1-292be2ba0a25 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:07.305194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16a052-10aa-4fa8-afd1-292be2ba0a25 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:07.305635 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16a052-10aa-4fa8-afd1-292be2ba0a25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:07.316400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16a052-10aa-4fa8-afd1-292be2ba0a25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:07.317860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16a052-10aa-4fa8-afd1-292be2ba0a25 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:07.336582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-bc16a052-10aa-4fa8-afd1-292be2ba0a25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:07.338587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:07.339593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:07.340524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:07.340524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:07.340524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:07.340524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:07.340524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 343/673] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:07.345476 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16a052-10aa-4fa8-afd1-292be2ba0a25 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:07.391624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88d943-053d-41e3-9e40-b9c098cecaf8 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:07.429510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd770f5-d7a8-4c12-9d0e-62a9c08d7c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:07.430170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd770f5-d7a8-4c12-9d0e-62a9c08d7c1c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:07.430450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd770f5-d7a8-4c12-9d0e-62a9c08d7c1c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:07.430721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd770f5-d7a8-4c12-9d0e-62a9c08d7c1c None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:07.431464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd770f5-d7a8-4c12-9d0e-62a9c08d7c1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:07.431767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd770f5-d7a8-4c12-9d0e-62a9c08d7c1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:07.433112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd770f5-d7a8-4c12-9d0e-62a9c08d7c1c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:07.449408 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 331/674] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:07.449408 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd770f5-d7a8-4c12-9d0e-62a9c08d7c1c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:07.458208 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:07.465309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16a052-10aa-4fa8-afd1-292be2ba0a25 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:07.502495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfa995-b32a-4e73-8cd6-48a3c4fbb47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:07.504797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfa995-b32a-4e73-8cd6-48a3c4fbb47e None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:07.505240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfa995-b32a-4e73-8cd6-48a3c4fbb47e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:07.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfa995-b32a-4e73-8cd6-48a3c4fbb47e None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:07.506303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfa995-b32a-4e73-8cd6-48a3c4fbb47e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:07.506611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfa995-b32a-4e73-8cd6-48a3c4fbb47e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:07.508060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfa995-b32a-4e73-8cd6-48a3c4fbb47e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:07.520720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 344/675] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:07.521541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfa995-b32a-4e73-8cd6-48a3c4fbb47e None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:07.558899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd770f5-d7a8-4c12-9d0e-62a9c08d7c1c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:07.574024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:07.594962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac00b24-e35d-43e9-b88e-1fe0c9ff7c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:07.595619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac00b24-e35d-43e9-b88e-1fe0c9ff7c18 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:07.595902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac00b24-e35d-43e9-b88e-1fe0c9ff7c18 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:07.596180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac00b24-e35d-43e9-b88e-1fe0c9ff7c18 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:07.597775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac00b24-e35d-43e9-b88e-1fe0c9ff7c18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:07.598089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac00b24-e35d-43e9-b88e-1fe0c9ff7c18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:07.599543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac00b24-e35d-43e9-b88e-1fe0c9ff7c18 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:07.613114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 332/676] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:07.614303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac00b24-e35d-43e9-b88e-1fe0c9ff7c18 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:07.634896 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfa995-b32a-4e73-8cd6-48a3c4fbb47e None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:07.670915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef73cd-e797-4600-95b6-38efbca6ce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:07.671696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef73cd-e797-4600-95b6-38efbca6ce5c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:07.671983 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef73cd-e797-4600-95b6-38efbca6ce5c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:07.672305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef73cd-e797-4600-95b6-38efbca6ce5c None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:07.672982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef73cd-e797-4600-95b6-38efbca6ce5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:07.673289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef73cd-e797-4600-95b6-38efbca6ce5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:07.675526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef73cd-e797-4600-95b6-38efbca6ce5c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:07.692689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 345/677] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:07.692689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef73cd-e797-4600-95b6-38efbca6ce5c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:07.710536 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:07.719085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac00b24-e35d-43e9-b88e-1fe0c9ff7c18 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:07.754739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb5cb9-dd53-4b6a-9469-268006676fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:07.755491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb5cb9-dd53-4b6a-9469-268006676fa9 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:07.755776 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb5cb9-dd53-4b6a-9469-268006676fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:07.756052 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb5cb9-dd53-4b6a-9469-268006676fa9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:07.763503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb5cb9-dd53-4b6a-9469-268006676fa9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:07.764986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb5cb9-dd53-4b6a-9469-268006676fa9 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-27cb5cb9-dd53-4b6a-9469-268006676fa9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:07.780876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:07.781890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:07.782766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:07.782766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:07.782766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:07.782766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:07.782766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:07.782766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:07.782766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:07.782766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:07.782766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 333/678] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:07.783298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb5cb9-dd53-4b6a-9469-268006676fa9 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:07.804563 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef73cd-e797-4600-95b6-38efbca6ce5c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:07.824294 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:07.841785 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921dc901-409f-43f2-a313-e2b457d0d7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:07.842703 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dc901-409f-43f2-a313-e2b457d0d7d1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:07.843185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dc901-409f-43f2-a313-e2b457d0d7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:07.843644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dc901-409f-43f2-a313-e2b457d0d7d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:07.844681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921dc901-409f-43f2-a313-e2b457d0d7d1 None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=1ed0973226a2446ebf7c0cc3ee04e446) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:07.845168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921dc901-409f-43f2-a313-e2b457d0d7d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:07.846776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921dc901-409f-43f2-a313-e2b457d0d7d1 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:07.853094 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 346/679] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:07 2022] GET /identity/v3/role_assignments?user.id=1ed0973226a2446ebf7c0cc3ee04e446&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:07.854073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921dc901-409f-43f2-a313-e2b457d0d7d1 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:07.910873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb5cb9-dd53-4b6a-9469-268006676fa9 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:07.960245 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8666b5f-c984-4ab5-a4b3-86eedc9a7ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:07.960923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8666b5f-c984-4ab5-a4b3-86eedc9a7ef4 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:07.961205 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8666b5f-c984-4ab5-a4b3-86eedc9a7ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:07.961477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8666b5f-c984-4ab5-a4b3-86eedc9a7ef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:07.962315 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8666b5f-c984-4ab5-a4b3-86eedc9a7ef4 None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=666ade4ce1fe447487998d086779389b) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:07.962626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8666b5f-c984-4ab5-a4b3-86eedc9a7ef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:07.964246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8666b5f-c984-4ab5-a4b3-86eedc9a7ef4 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:07.970927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 334/680] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:07 2022] GET /identity/v3/role_assignments?user.id=666ade4ce1fe447487998d086779389b&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:07.971650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8666b5f-c984-4ab5-a4b3-86eedc9a7ef4 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:08.001633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:08.029289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921dc901-409f-43f2-a313-e2b457d0d7d1 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:08.075674 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364894ed-0c4d-4685-9319-4bf21c980e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:08.076929 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364894ed-0c4d-4685-9319-4bf21c980e53 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:08.077421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364894ed-0c4d-4685-9319-4bf21c980e53 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:08.077702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364894ed-0c4d-4685-9319-4bf21c980e53 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:08.089825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364894ed-0c4d-4685-9319-4bf21c980e53 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:08.091884 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364894ed-0c4d-4685-9319-4bf21c980e53 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-364894ed-0c4d-4685-9319-4bf21c980e53 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:08.110036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:08.111323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:08.112334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:08.112334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:08.112334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:08.112334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:08.112334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:08.112334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:08.112334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:08.112334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:08.112334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 13:41:08.112334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:08.117599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 347/681] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:41:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:08.119780 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02390a90-10dd-4ddc-b95c-628797971308 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:08.119922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:08.120018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02390a90-10dd-4ddc-b95c-628797971308 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:08.120018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02390a90-10dd-4ddc-b95c-628797971308 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:08.122296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 348/682] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:08.122465 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02390a90-10dd-4ddc-b95c-628797971308 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:08.127682 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8666b5f-c984-4ab5-a4b3-86eedc9a7ef4 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:08.176947 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bee998-290f-40bd-b5af-d7a176e20143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:08.177623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bee998-290f-40bd-b5af-d7a176e20143 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:08.177908 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bee998-290f-40bd-b5af-d7a176e20143 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:08.178185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bee998-290f-40bd-b5af-d7a176e20143 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:08.179139 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bee998-290f-40bd-b5af-d7a176e20143 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:08.179456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bee998-290f-40bd-b5af-d7a176e20143 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:08.181106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bee998-290f-40bd-b5af-d7a176e20143 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:08.198903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 335/683] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:08.198903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848bd07-3856-4c9e-91ff-18bed913a7d7 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:08.198903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848bd07-3856-4c9e-91ff-18bed913a7d7 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:08.198903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848bd07-3856-4c9e-91ff-18bed913a7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:08.217552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-0848bd07-3856-4c9e-91ff-18bed913a7d7 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:08.218811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-0848bd07-3856-4c9e-91ff-18bed913a7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:08.265485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02390a90-10dd-4ddc-b95c-628797971308 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:08.299608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 336/684] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:41:08.300568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848bd07-3856-4c9e-91ff-18bed913a7d7 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:08.303636 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f3b74-3334-4ced-8684-2bf2c756722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:08.304571 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f3b74-3334-4ced-8684-2bf2c756722c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:08.305080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f3b74-3334-4ced-8684-2bf2c756722c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:08.305554 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f3b74-3334-4ced-8684-2bf2c756722c None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:08.306602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f3b74-3334-4ced-8684-2bf2c756722c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:08.307145 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f3b74-3334-4ced-8684-2bf2c756722c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:08.308793 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f3b74-3334-4ced-8684-2bf2c756722c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:08.327215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:08.339357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 349/685] 10.222.0.52 () {56 vars in 1135 bytes} [Thu May 19 13:41:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:08.340294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f3b74-3334-4ced-8684-2bf2c756722c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:08.413391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:08.425559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848bd07-3856-4c9e-91ff-18bed913a7d7 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:08.450697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f3b74-3334-4ced-8684-2bf2c756722c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:08.461441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afa9e2-9276-4bbc-99a8-0fb4d952e788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:08.462109 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afa9e2-9276-4bbc-99a8-0fb4d952e788 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:08.462389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afa9e2-9276-4bbc-99a8-0fb4d952e788 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:08.462665 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afa9e2-9276-4bbc-99a8-0fb4d952e788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:08.463637 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afa9e2-9276-4bbc-99a8-0fb4d952e788 None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=f18f635bff0142409e90da66547927f1) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:08.464004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afa9e2-9276-4bbc-99a8-0fb4d952e788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:08.465506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afa9e2-9276-4bbc-99a8-0fb4d952e788 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:08.472100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 337/686] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:08 2022] GET /identity/v3/role_assignments?user.id=f18f635bff0142409e90da66547927f1&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:08.473152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afa9e2-9276-4bbc-99a8-0fb4d952e788 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:08.489305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367e1d7-f0d7-4a0e-9dcd-4cce4bad2806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:08.490114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367e1d7-f0d7-4a0e-9dcd-4cce4bad2806 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:08.490511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367e1d7-f0d7-4a0e-9dcd-4cce4bad2806 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:08.490801 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367e1d7-f0d7-4a0e-9dcd-4cce4bad2806 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:08.505475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367e1d7-f0d7-4a0e-9dcd-4cce4bad2806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-d367e1d7-f0d7-4a0e-9dcd-4cce4bad2806 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:08.513602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:08.514585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:08.515467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:08.515467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:08.515467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:08.515467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:08.515467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:08.515467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:08.515467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:08.515467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:08.515467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:08.515467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:08.515467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:08.515467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:08.515467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 350/687] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:41:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:08.521734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367e1d7-f0d7-4a0e-9dcd-4cce4bad2806 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:08.633349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:08.667946 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afa9e2-9276-4bbc-99a8-0fb4d952e788 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:08.692835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367e1d7-f0d7-4a0e-9dcd-4cce4bad2806 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:08.729648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17397094-c74f-48ba-9596-2f4ef14ea31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:08.730666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17397094-c74f-48ba-9596-2f4ef14ea31d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:08.731194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17397094-c74f-48ba-9596-2f4ef14ea31d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:08.731705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17397094-c74f-48ba-9596-2f4ef14ea31d None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:08.732465 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17397094-c74f-48ba-9596-2f4ef14ea31d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:08.732769 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17397094-c74f-48ba-9596-2f4ef14ea31d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:08.734148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17397094-c74f-48ba-9596-2f4ef14ea31d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:08.745215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 351/688] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:41:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:08.747471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d590ce2-d68e-4c40-ad93-7fbd04f2727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:08.748298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d590ce2-d68e-4c40-ad93-7fbd04f2727c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:08.748586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d590ce2-d68e-4c40-ad93-7fbd04f2727c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:08.748922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d590ce2-d68e-4c40-ad93-7fbd04f2727c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:08.750529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17397094-c74f-48ba-9596-2f4ef14ea31d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:08.777655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:08.841370 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d590ce2-d68e-4c40-ad93-7fbd04f2727c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:08.844054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d590ce2-d68e-4c40-ad93-7fbd04f2727c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:08.876402 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17397094-c74f-48ba-9596-2f4ef14ea31d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:08.908947 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:08.928127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded727c9-3da0-4498-862c-92b115bcfade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:08.929260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded727c9-3da0-4498-862c-92b115bcfade None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:08.929746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded727c9-3da0-4498-862c-92b115bcfade None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:08.930220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded727c9-3da0-4498-862c-92b115bcfade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:08.943193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded727c9-3da0-4498-862c-92b115bcfade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:08.944963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded727c9-3da0-4498-862c-92b115bcfade None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-ded727c9-3da0-4498-862c-92b115bcfade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:08.964270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:08.965198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:08.966121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:08.966121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:08.966121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:08.966121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:08.969083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 352/689] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:08.974260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded727c9-3da0-4498-862c-92b115bcfade None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:08.979757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 338/690] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:41:08.984338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d590ce2-d68e-4c40-ad93-7fbd04f2727c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:09.073609 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:09.146836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded727c9-3da0-4498-862c-92b115bcfade None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:09.155241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d590ce2-d68e-4c40-ad93-7fbd04f2727c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:09.206057 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c07ed-d34f-4216-ab8f-302f6052e924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:09.207213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c07ed-d34f-4216-ab8f-302f6052e924 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:09.207689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c07ed-d34f-4216-ab8f-302f6052e924 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:09.207969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c07ed-d34f-4216-ab8f-302f6052e924 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:09.208694 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c07ed-d34f-4216-ab8f-302f6052e924 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:09.208998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c07ed-d34f-4216-ab8f-302f6052e924 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:09.210358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c07ed-d34f-4216-ab8f-302f6052e924 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:09.225180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabd785-eb8e-4e5e-b2e8-c0e38c54a4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:09.225980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabd785-eb8e-4e5e-b2e8-c0e38c54a4d5 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:09.226259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabd785-eb8e-4e5e-b2e8-c0e38c54a4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:09.226527 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabd785-eb8e-4e5e-b2e8-c0e38c54a4d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:09.237679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 353/691] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:09.238632 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c07ed-d34f-4216-ab8f-302f6052e924 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:09.245307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceabd785-eb8e-4e5e-b2e8-c0e38c54a4d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:09.247035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceabd785-eb8e-4e5e-b2e8-c0e38c54a4d5 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-ceabd785-eb8e-4e5e-b2e8-c0e38c54a4d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:09.276246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:09.277208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:09.278179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:09.278179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:09.278179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:09.278179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:09.278179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 339/692] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:09.283069 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabd785-eb8e-4e5e-b2e8-c0e38c54a4d5 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:09.298581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:09.376603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c07ed-d34f-4216-ab8f-302f6052e924 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:09.551051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f69fe4-0da2-4b86-8ce2-b659480e7613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:09.552844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f69fe4-0da2-4b86-8ce2-b659480e7613 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:09.553820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f69fe4-0da2-4b86-8ce2-b659480e7613 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:09.554428 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f69fe4-0da2-4b86-8ce2-b659480e7613 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:09.556423 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabd785-eb8e-4e5e-b2e8-c0e38c54a4d5 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:09.561634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:09.575732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f69fe4-0da2-4b86-8ce2-b659480e7613 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:09.577214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f69fe4-0da2-4b86-8ce2-b659480e7613 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:09.599890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2804416-0a28-474f-bb6d-c85b78d88bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:09.600827 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2804416-0a28-474f-bb6d-c85b78d88bdd None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:09.601263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2804416-0a28-474f-bb6d-c85b78d88bdd None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:09.601685 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2804416-0a28-474f-bb6d-c85b78d88bdd None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:09.602550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2804416-0a28-474f-bb6d-c85b78d88bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:09.603044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2804416-0a28-474f-bb6d-c85b78d88bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:09.604544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2804416-0a28-474f-bb6d-c85b78d88bdd None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-71f69fe4-0da2-4b86-8ce2-b659480e7613 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:09.607126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:09.608036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:09.608974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:09.608974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:09.608974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:09.608974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:09.608974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:09.608974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:09.608974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:09.608974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:09.608974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:41:09.608974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:09.614996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 354/693] 10.222.0.52 () {56 vars in 1093 bytes} [Thu May 19 13:41:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:09.620483 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f69fe4-0da2-4b86-8ce2-b659480e7613 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:09.621973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 340/694] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:09.627785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2804416-0a28-474f-bb6d-c85b78d88bdd None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:09.824261 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2804416-0a28-474f-bb6d-c85b78d88bdd None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:09.832844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:09.870907 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f69fe4-0da2-4b86-8ce2-b659480e7613 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:09.879674 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c097fd-50f1-433f-8694-8191d8011c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:09.880660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c097fd-50f1-433f-8694-8191d8011c96 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:09.881088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c097fd-50f1-433f-8694-8191d8011c96 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:09.881502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c097fd-50f1-433f-8694-8191d8011c96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:09.897361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c097fd-50f1-433f-8694-8191d8011c96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:09.899099 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c097fd-50f1-433f-8694-8191d8011c96 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-a7c097fd-50f1-433f-8694-8191d8011c96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:09.928917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:09.930063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:09.931183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:09.931183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:09.931183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:09.931183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:09.931183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:09.931183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:09.931183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:09.931183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:09.935878 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 341/695] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:09.943941 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c097fd-50f1-433f-8694-8191d8011c96 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:09.952173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e1442-e2c7-41bf-bf96-63aabdab60d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:09.953445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e1442-e2c7-41bf-bf96-63aabdab60d7 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:09.953934 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e1442-e2c7-41bf-bf96-63aabdab60d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:09.954386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e1442-e2c7-41bf-bf96-63aabdab60d7 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:09.955606 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e1442-e2c7-41bf-bf96-63aabdab60d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:09.956134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e1442-e2c7-41bf-bf96-63aabdab60d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:09.958378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e1442-e2c7-41bf-bf96-63aabdab60d7 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:09.987580 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 355/696] 10.222.0.52 () {56 vars in 1133 bytes} [Thu May 19 13:41:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:09.992728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e1442-e2c7-41bf-bf96-63aabdab60d7 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:09.996235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:10.137217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c097fd-50f1-433f-8694-8191d8011c96 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:10.164134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e1442-e2c7-41bf-bf96-63aabdab60d7 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:10.204054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6244cc-6e0d-4bcc-8737-77a3a0ca7142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:10.204919 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6244cc-6e0d-4bcc-8737-77a3a0ca7142 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.205320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6244cc-6e0d-4bcc-8737-77a3a0ca7142 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.205697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6244cc-6e0d-4bcc-8737-77a3a0ca7142 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.206894 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b8a48-5dfd-4c5c-90ef-9aeaccce77c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:10.207596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b8a48-5dfd-4c5c-90ef-9aeaccce77c7 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.207879 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b8a48-5dfd-4c5c-90ef-9aeaccce77c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.208155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b8a48-5dfd-4c5c-90ef-9aeaccce77c7 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.208947 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b8a48-5dfd-4c5c-90ef-9aeaccce77c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:10.209254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b8a48-5dfd-4c5c-90ef-9aeaccce77c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:10.210678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b8a48-5dfd-4c5c-90ef-9aeaccce77c7 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:10.228149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:10.232762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 342/697] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:10.239178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b8a48-5dfd-4c5c-90ef-9aeaccce77c7 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:10.292819 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6244cc-6e0d-4bcc-8737-77a3a0ca7142 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:10.294292 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6244cc-6e0d-4bcc-8737-77a3a0ca7142 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:10.429932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 356/698] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:41:10.435034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6244cc-6e0d-4bcc-8737-77a3a0ca7142 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:10.453200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:10.473431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b8a48-5dfd-4c5c-90ef-9aeaccce77c7 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:10.517753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c6f64-acb0-486f-9e81-1311b0ceca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:10.519924 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c6f64-acb0-486f-9e81-1311b0ceca99 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.520489 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c6f64-acb0-486f-9e81-1311b0ceca99 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.520974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c6f64-acb0-486f-9e81-1311b0ceca99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.522013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c6f64-acb0-486f-9e81-1311b0ceca99 None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=8554ff40bbcb4fbb801d854d6527a6d2) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:10.522564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c6f64-acb0-486f-9e81-1311b0ceca99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:10.524243 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c6f64-acb0-486f-9e81-1311b0ceca99 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:10.525197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:10.531176 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 343/699] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:10 2022] GET /identity/v3/role_assignments?user.id=8554ff40bbcb4fbb801d854d6527a6d2&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:10.532689 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde9f2a-1db3-409a-a674-db45a4cd6828 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.533050 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde9f2a-1db3-409a-a674-db45a4cd6828 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.533406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde9f2a-1db3-409a-a674-db45a4cd6828 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.540560 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 344/700] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:10.542191 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c538d3-802f-4b77-ac74-1cac0abae4d0 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.542565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c538d3-802f-4b77-ac74-1cac0abae4d0 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.542947 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c538d3-802f-4b77-ac74-1cac0abae4d0 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.544764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 345/701] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:10.546130 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec060e-b04a-4ca7-af69-f58ef9c0b053 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.546512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec060e-b04a-4ca7-af69-f58ef9c0b053 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.546884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec060e-b04a-4ca7-af69-f58ef9c0b053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.591240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6244cc-6e0d-4bcc-8737-77a3a0ca7142 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:10.635584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17686ffd-e8ec-499e-93fb-e3c9e82cd1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:10.636718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17686ffd-e8ec-499e-93fb-e3c9e82cd1a7 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.637182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17686ffd-e8ec-499e-93fb-e3c9e82cd1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.637634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17686ffd-e8ec-499e-93fb-e3c9e82cd1a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.651050 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17686ffd-e8ec-499e-93fb-e3c9e82cd1a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:10.652867 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17686ffd-e8ec-499e-93fb-e3c9e82cd1a7 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-17686ffd-e8ec-499e-93fb-e3c9e82cd1a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:10.674069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:10.676113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:10.677107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:10.677107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:10.677107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:10.677107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:10.679288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 357/702] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:10.680991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4212c-2d7e-4adf-9736-c1a2ea88a151 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.681423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4212c-2d7e-4adf-9736-c1a2ea88a151 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.681854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4212c-2d7e-4adf-9736-c1a2ea88a151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.701327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-74c4212c-2d7e-4adf-9736-c1a2ea88a151 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:10.702776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-74c4212c-2d7e-4adf-9736-c1a2ea88a151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:10.703085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-9fec060e-b04a-4ca7-af69-f58ef9c0b053 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:10.704466 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9fec060e-b04a-4ca7-af69-f58ef9c0b053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:10.783725 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 358/703] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:41:10.785082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4212c-2d7e-4adf-9736-c1a2ea88a151 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:10.802988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 346/704] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:41:10.810940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec060e-b04a-4ca7-af69-f58ef9c0b053 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:10.821412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:10.908137 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4212c-2d7e-4adf-9736-c1a2ea88a151 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:10.915278 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:10.947708 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec060e-b04a-4ca7-af69-f58ef9c0b053 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:10.951014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800eb974-ad5a-4e5b-b3f9-f3a8cb90bd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:10.951700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800eb974-ad5a-4e5b-b3f9-f3a8cb90bd38 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.951987 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800eb974-ad5a-4e5b-b3f9-f3a8cb90bd38 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.952262 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800eb974-ad5a-4e5b-b3f9-f3a8cb90bd38 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.952996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800eb974-ad5a-4e5b-b3f9-f3a8cb90bd38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:10.953305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800eb974-ad5a-4e5b-b3f9-f3a8cb90bd38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:10.954675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800eb974-ad5a-4e5b-b3f9-f3a8cb90bd38 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:10.969814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 359/705] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:10.975148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800eb974-ad5a-4e5b-b3f9-f3a8cb90bd38 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:10.996370 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb67ee-9a00-43be-9f2b-e2dd12a851ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:10.997848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb67ee-9a00-43be-9f2b-e2dd12a851ec None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.998447 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb67ee-9a00-43be-9f2b-e2dd12a851ec None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.998741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb67ee-9a00-43be-9f2b-e2dd12a851ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:11.012238 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb67ee-9a00-43be-9f2b-e2dd12a851ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:11.014388 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb67ee-9a00-43be-9f2b-e2dd12a851ec None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-f5bb67ee-9a00-43be-9f2b-e2dd12a851ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:11.037226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:11.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:11.039087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:11.039087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:11.039087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:11.039087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:11.039324 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 347/706] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:11.042883 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb67ee-9a00-43be-9f2b-e2dd12a851ec None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:11.083118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:11.094585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800eb974-ad5a-4e5b-b3f9-f3a8cb90bd38 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:11.135172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23f4ff-5c2a-42d4-89d6-5d056a6c347e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:11.136332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23f4ff-5c2a-42d4-89d6-5d056a6c347e None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:11.136842 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23f4ff-5c2a-42d4-89d6-5d056a6c347e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:11.137294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23f4ff-5c2a-42d4-89d6-5d056a6c347e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:11.138152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23f4ff-5c2a-42d4-89d6-5d056a6c347e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:11.139732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23f4ff-5c2a-42d4-89d6-5d056a6c347e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-ad23f4ff-5c2a-42d4-89d6-5d056a6c347e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:11.154366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:11.155411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:41:11.156241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:11.164385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 360/707] 10.222.0.52 () {56 vars in 1129 bytes} [Thu May 19 13:41:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:11.165600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23f4ff-5c2a-42d4-89d6-5d056a6c347e None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:11.167656 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb67ee-9a00-43be-9f2b-e2dd12a851ec None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:11.205065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:11.213625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27663406-594d-45ee-bc08-2c2e9484e976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:11.214426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27663406-594d-45ee-bc08-2c2e9484e976 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:11.214920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27663406-594d-45ee-bc08-2c2e9484e976 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:11.216089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27663406-594d-45ee-bc08-2c2e9484e976 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:11.226744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27663406-594d-45ee-bc08-2c2e9484e976 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:11.230196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27663406-594d-45ee-bc08-2c2e9484e976 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-27663406-594d-45ee-bc08-2c2e9484e976 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:11.246413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:11.247666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:11.248535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:11.248535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:11.248535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:11.248535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:11.248535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:11.248535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:11.248535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:11.248535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:11.251387 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 348/708] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:11.252588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27663406-594d-45ee-bc08-2c2e9484e976 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:11.292742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23f4ff-5c2a-42d4-89d6-5d056a6c347e None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:11.337110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc01ad1-b180-4bbd-bfbd-7ff287cee397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:11.337787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc01ad1-b180-4bbd-bfbd-7ff287cee397 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:11.338072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc01ad1-b180-4bbd-bfbd-7ff287cee397 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:11.338350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc01ad1-b180-4bbd-bfbd-7ff287cee397 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:11.339109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc01ad1-b180-4bbd-bfbd-7ff287cee397 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:11.339420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc01ad1-b180-4bbd-bfbd-7ff287cee397 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:11.340797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc01ad1-b180-4bbd-bfbd-7ff287cee397 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:11.360208 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 361/709] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:11.361575 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc01ad1-b180-4bbd-bfbd-7ff287cee397 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:11.374575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27663406-594d-45ee-bc08-2c2e9484e976 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:11.381930 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:11.419296 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4522f5-f86d-4cf2-ba20-315a62108b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:11.420042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4522f5-f86d-4cf2-ba20-315a62108b4a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:11.420347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4522f5-f86d-4cf2-ba20-315a62108b4a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:11.420700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4522f5-f86d-4cf2-ba20-315a62108b4a None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:11.421695 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4522f5-f86d-4cf2-ba20-315a62108b4a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:11.422019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4522f5-f86d-4cf2-ba20-315a62108b4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:11.423456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4522f5-f86d-4cf2-ba20-315a62108b4a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:11.433991 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 349/710] 10.222.0.52 () {56 vars in 1124 bytes} [Thu May 19 13:41:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:11.433991 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4522f5-f86d-4cf2-ba20-315a62108b4a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:11.488145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:11.494934 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc01ad1-b180-4bbd-bfbd-7ff287cee397 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:11.530749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ef5a9-3d1d-4fd9-bb6c-0bf69b6abd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:11.531443 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef5a9-3d1d-4fd9-bb6c-0bf69b6abd85 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:11.531732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef5a9-3d1d-4fd9-bb6c-0bf69b6abd85 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:11.532011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef5a9-3d1d-4fd9-bb6c-0bf69b6abd85 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:11.532800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ef5a9-3d1d-4fd9-bb6c-0bf69b6abd85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:11.533115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ef5a9-3d1d-4fd9-bb6c-0bf69b6abd85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:11.535220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ef5a9-3d1d-4fd9-bb6c-0bf69b6abd85 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:11.535536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4522f5-f86d-4cf2-ba20-315a62108b4a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:11.555158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 362/711] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:11.555158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ef5a9-3d1d-4fd9-bb6c-0bf69b6abd85 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:11.576459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ec2a0-5a44-4e5f-8e80-f632e9c6426e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:11.577128 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ec2a0-5a44-4e5f-8e80-f632e9c6426e None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:11.577416 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ec2a0-5a44-4e5f-8e80-f632e9c6426e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:11.577696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ec2a0-5a44-4e5f-8e80-f632e9c6426e None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:11.578397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ec2a0-5a44-4e5f-8e80-f632e9c6426e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:11.579859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ec2a0-5a44-4e5f-8e80-f632e9c6426e None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:11.610888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 350/712] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:41:11 2022] POST /identity/v3/users => generated 254 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:11.610888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06383-2859-46e9-a8a4-f96043ff2e4f None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:11.610888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06383-2859-46e9-a8a4-f96043ff2e4f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:11.610888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06383-2859-46e9-a8a4-f96043ff2e4f None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:11.610888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 351/713] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:11.610888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b06383-2859-46e9-a8a4-f96043ff2e4f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:11.654385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:11.690208 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ef5a9-3d1d-4fd9-bb6c-0bf69b6abd85 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:11.741815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0f1ac-c77f-4de8-b524-afc93443e60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:11.742660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0f1ac-c77f-4de8-b524-afc93443e60d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:11.743110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0f1ac-c77f-4de8-b524-afc93443e60d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:11.743393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0f1ac-c77f-4de8-b524-afc93443e60d None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:11.744290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0f1ac-c77f-4de8-b524-afc93443e60d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:11.744603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0f1ac-c77f-4de8-b524-afc93443e60d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:11.746886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0f1ac-c77f-4de8-b524-afc93443e60d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:11.752325 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b06383-2859-46e9-a8a4-f96043ff2e4f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:11.755513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 363/714] 10.222.0.52 () {56 vars in 1124 bytes} [Thu May 19 13:41:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:11.757814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe39caa-dd40-4a8e-97fe-7f5c025e8fae None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:11.758106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe39caa-dd40-4a8e-97fe-7f5c025e8fae None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:11.758388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe39caa-dd40-4a8e-97fe-7f5c025e8fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:11.767278 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:11.780691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-5fe39caa-dd40-4a8e-97fe-7f5c025e8fae None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:11.782582 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5fe39caa-dd40-4a8e-97fe-7f5c025e8fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:11.812867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414b3f9-05f0-414b-acb0-40fbe27efab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:11.813602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414b3f9-05f0-414b-acb0-40fbe27efab8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:11.813893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414b3f9-05f0-414b-acb0-40fbe27efab8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:11.814169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414b3f9-05f0-414b-acb0-40fbe27efab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:11.815150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414b3f9-05f0-414b-acb0-40fbe27efab8 None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=4f6093d0c1fc41d98c58bace81e3d8e5) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:11.816093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414b3f9-05f0-414b-acb0-40fbe27efab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:11.818040 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414b3f9-05f0-414b-acb0-40fbe27efab8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:11.825522 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 352/715] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:11 2022] GET /identity/v3/role_assignments?user.id=4f6093d0c1fc41d98c58bace81e3d8e5&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=0893bb944dd94634accd3906aa391cd2 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:11.903360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 364/716] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:41:11.927506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414b3f9-05f0-414b-acb0-40fbe27efab8 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:12.069347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414b3f9-05f0-414b-acb0-40fbe27efab8 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:12.109946 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f8130-4532-4b94-8ea1-d7e310f9745d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:12.111168 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f8130-4532-4b94-8ea1-d7e310f9745d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:12.111676 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f8130-4532-4b94-8ea1-d7e310f9745d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:12.112146 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f8130-4532-4b94-8ea1-d7e310f9745d None admin] PATH_INFO: `/v3/services/network` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:12.113577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f8130-4532-4b94-8ea1-d7e310f9745d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:12.115173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f8130-4532-4b94-8ea1-d7e310f9745d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-737f8130-4532-4b94-8ea1-d7e310f9745d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:12.134885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:12.135971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:41:12.136803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:12.139416 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 353/717] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:11 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:12.145622 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe39caa-dd40-4a8e-97fe-7f5c025e8fae None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:12.245154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:12.290784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe39caa-dd40-4a8e-97fe-7f5c025e8fae None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:12.330524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cf071-fc50-4fde-ba59-87cb0ba8c5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:12.331182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf071-fc50-4fde-ba59-87cb0ba8c5a0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:12.331469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf071-fc50-4fde-ba59-87cb0ba8c5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:12.331743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf071-fc50-4fde-ba59-87cb0ba8c5a0 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:12.332460 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cf071-fc50-4fde-ba59-87cb0ba8c5a0 None admin] RBAC: Adding query filter params (name=network) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:12.332764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cf071-fc50-4fde-ba59-87cb0ba8c5a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:12.334132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cf071-fc50-4fde-ba59-87cb0ba8c5a0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:12.339270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 365/718] 10.222.0.52 () {56 vars in 1112 bytes} [Thu May 19 13:41:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:12.344447 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f8130-4532-4b94-8ea1-d7e310f9745d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:12.412546 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:12.480313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f8130-4532-4b94-8ea1-d7e310f9745d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:12.522727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a328929b-7d7a-4110-b2a7-165f6a6cd8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:12.523429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328929b-7d7a-4110-b2a7-165f6a6cd8c0 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:12.523713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328929b-7d7a-4110-b2a7-165f6a6cd8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:12.523988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328929b-7d7a-4110-b2a7-165f6a6cd8c0 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:12.524696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328929b-7d7a-4110-b2a7-165f6a6cd8c0 None admin] RBAC: Adding query filter params (type=network) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:12.524999 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328929b-7d7a-4110-b2a7-165f6a6cd8c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:12.526359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328929b-7d7a-4110-b2a7-165f6a6cd8c0 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:12.532372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 354/719] 10.222.0.52 () {56 vars in 1112 bytes} [Thu May 19 13:41:12 2022] GET /identity/v3/services?type=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:13.202307 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915510d5-9b5b-45e1-b305-cf32b763faa5 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:13.202839 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915510d5-9b5b-45e1-b305-cf32b763faa5 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:13.202950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915510d5-9b5b-45e1-b305-cf32b763faa5 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:13.204586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 366/720] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:13.216752 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641ea56-ee4e-4c7f-aca8-0ba8a61ec19b None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:13.217217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641ea56-ee4e-4c7f-aca8-0ba8a61ec19b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:13.217292 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641ea56-ee4e-4c7f-aca8-0ba8a61ec19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:13.240933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-9641ea56-ee4e-4c7f-aca8-0ba8a61ec19b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:13.242166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9641ea56-ee4e-4c7f-aca8-0ba8a61ec19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:13.328455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 355/721] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:41:13.359857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915510d5-9b5b-45e1-b305-cf32b763faa5 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:13.504951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915510d5-9b5b-45e1-b305-cf32b763faa5 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:13.545159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20f819-5ee5-401a-8cfc-6709f588146a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:13.545951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20f819-5ee5-401a-8cfc-6709f588146a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:13.546237 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20f819-5ee5-401a-8cfc-6709f588146a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:13.546517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20f819-5ee5-401a-8cfc-6709f588146a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:13.548371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:13.558799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20f819-5ee5-401a-8cfc-6709f588146a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-5b20f819-5ee5-401a-8cfc-6709f588146a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:13.566440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:13.567397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:13.568294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:13.568294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:13.568294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:13.568294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:13.568294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:13.568294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:13.568294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:13.568294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:13.568294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:13.568294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:13.568294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:13.568294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:13.568294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 367/722] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:41:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:13.575339 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641ea56-ee4e-4c7f-aca8-0ba8a61ec19b None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:13.616510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:13.712838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641ea56-ee4e-4c7f-aca8-0ba8a61ec19b None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:13.748842 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd915c6-0adc-464c-a851-e8708f970d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:13.749779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd915c6-0adc-464c-a851-e8708f970d3a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:13.750207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd915c6-0adc-464c-a851-e8708f970d3a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:13.750625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd915c6-0adc-464c-a851-e8708f970d3a None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:13.751606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd915c6-0adc-464c-a851-e8708f970d3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:13.752076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd915c6-0adc-464c-a851-e8708f970d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:13.753599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd915c6-0adc-464c-a851-e8708f970d3a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:13.762910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 356/723] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:41:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:13.770887 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20f819-5ee5-401a-8cfc-6709f588146a None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:13.942289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20f819-5ee5-401a-8cfc-6709f588146a None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:13.997711 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dca52-7907-4286-90e5-c799d0c2ef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:13.998504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dca52-7907-4286-90e5-c799d0c2ef7d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:13.998788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dca52-7907-4286-90e5-c799d0c2ef7d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:13.999086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dca52-7907-4286-90e5-c799d0c2ef7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:14.022493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532dca52-7907-4286-90e5-c799d0c2ef7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:14.024601 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532dca52-7907-4286-90e5-c799d0c2ef7d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-532dca52-7907-4286-90e5-c799d0c2ef7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:14.048618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:14.049648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:14.050734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:14.050734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:14.050734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:14.050734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:14.050734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 368/724] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:14.056103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd915c6-0adc-464c-a851-e8708f970d3a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:14.232796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:14.242324 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd915c6-0adc-464c-a851-e8708f970d3a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:14.288709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11091d4-3c9e-4a7b-8a0c-ea88f530ef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:14.289289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11091d4-3c9e-4a7b-8a0c-ea88f530ef50 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:14.289574 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11091d4-3c9e-4a7b-8a0c-ea88f530ef50 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:14.289847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11091d4-3c9e-4a7b-8a0c-ea88f530ef50 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:14.290572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11091d4-3c9e-4a7b-8a0c-ea88f530ef50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:14.290890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11091d4-3c9e-4a7b-8a0c-ea88f530ef50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:14.292252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11091d4-3c9e-4a7b-8a0c-ea88f530ef50 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:14.309549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 357/725] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:14.314529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dca52-7907-4286-90e5-c799d0c2ef7d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:14.421209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:14.522498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dca52-7907-4286-90e5-c799d0c2ef7d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:14.566131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e824db-8c02-4af0-bffc-1f392226cedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:14.566843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e824db-8c02-4af0-bffc-1f392226cedf None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:14.567149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e824db-8c02-4af0-bffc-1f392226cedf None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:14.567431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e824db-8c02-4af0-bffc-1f392226cedf None admin] PATH_INFO: `/v3/users/swift` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:14.580020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e824db-8c02-4af0-bffc-1f392226cedf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:14.582045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e824db-8c02-4af0-bffc-1f392226cedf None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-42e824db-8c02-4af0-bffc-1f392226cedf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:14.600396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:14.601370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:14.602283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:14.602283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:14.602283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:14.602283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:14.602283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:14.602283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:14.602283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:14.602283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:14.602283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 13:41:14.602283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:14.602283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 369/726] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:41:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:14.608911 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11091d4-3c9e-4a7b-8a0c-ea88f530ef50 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:14.618942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29dcab-a75b-40bf-8bc0-13770134b61f None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:14.619227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29dcab-a75b-40bf-8bc0-13770134b61f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:14.619507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29dcab-a75b-40bf-8bc0-13770134b61f None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:14.621165 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 370/727] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:41:14.628530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b62c-a65a-43ef-8dbb-1477b6c8a3b4 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:14.628842 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b62c-a65a-43ef-8dbb-1477b6c8a3b4 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:14.629346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b62c-a65a-43ef-8dbb-1477b6c8a3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:14.647084 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-44c0b62c-a65a-43ef-8dbb-1477b6c8a3b4 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:14.648286 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-44c0b62c-a65a-43ef-8dbb-1477b6c8a3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:14.726979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 371/728] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:41:14.728413 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32efc7-626f-4b4a-8b41-f077ee343a23 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:14.728710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32efc7-626f-4b4a-8b41-f077ee343a23 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:14.728993 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32efc7-626f-4b4a-8b41-f077ee343a23 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:14.730436 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 372/729] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:14.736148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dccf26-9912-4c5f-9e9b-a22702807d09 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:14.736440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dccf26-9912-4c5f-9e9b-a22702807d09 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:14.736727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dccf26-9912-4c5f-9e9b-a22702807d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:14.758061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11091d4-3c9e-4a7b-8a0c-ea88f530ef50 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:14.762724 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-d6dccf26-9912-4c5f-9e9b-a22702807d09 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:14.763998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d6dccf26-9912-4c5f-9e9b-a22702807d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:14.796051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:14.797009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed605d52-3faa-4c01-8540-3401b40c5f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:14.797799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed605d52-3faa-4c01-8540-3401b40c5f44 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:14.797976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed605d52-3faa-4c01-8540-3401b40c5f44 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:14.798258 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed605d52-3faa-4c01-8540-3401b40c5f44 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:14.799202 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed605d52-3faa-4c01-8540-3401b40c5f44 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:14.799516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed605d52-3faa-4c01-8540-3401b40c5f44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:14.801272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed605d52-3faa-4c01-8540-3401b40c5f44 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:14.821994 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 358/730] 10.222.0.52 () {56 vars in 1135 bytes} [Thu May 19 13:41:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:14.823208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41925c4e-90a9-4c68-8080-4fab7b2a6e19 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:14.823494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41925c4e-90a9-4c68-8080-4fab7b2a6e19 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:14.823766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41925c4e-90a9-4c68-8080-4fab7b2a6e19 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:14.826788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 359/731] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:14.826788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41925c4e-90a9-4c68-8080-4fab7b2a6e19 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:14.843072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:14.847886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 373/732] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:41:14.852720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dccf26-9912-4c5f-9e9b-a22702807d09 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:14.936547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41925c4e-90a9-4c68-8080-4fab7b2a6e19 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:14.965876 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dccf26-9912-4c5f-9e9b-a22702807d09 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:14.969724 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951680f-afa9-460e-ada3-b854baf07108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:14.970516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951680f-afa9-460e-ada3-b854baf07108 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:14.970795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951680f-afa9-460e-ada3-b854baf07108 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:14.971092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951680f-afa9-460e-ada3-b854baf07108 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:14.981094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951680f-afa9-460e-ada3-b854baf07108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-1951680f-afa9-460e-ada3-b854baf07108 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:14.989236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:14.990144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:14.991004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:14.991004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:14.991004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:14.991004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:14.991004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:14.991004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:14.991004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:14.991004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:14.991004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:14.991004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:14.991004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:14.991004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:14.991004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 360/733] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:41:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:14.991684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf8312-16ba-4ca4-a608-d86f0482d722 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:14.992244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf8312-16ba-4ca4-a608-d86f0482d722 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:14.992244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf8312-16ba-4ca4-a608-d86f0482d722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:15.009141 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-2baf8312-16ba-4ca4-a608-d86f0482d722 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:15.010304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2baf8312-16ba-4ca4-a608-d86f0482d722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:15.012863 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedff40b-9321-447d-b531-f1f1393e5716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:15.014593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedff40b-9321-447d-b531-f1f1393e5716 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:15.015101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedff40b-9321-447d-b531-f1f1393e5716 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:15.015556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedff40b-9321-447d-b531-f1f1393e5716 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:15.059413 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:15.078678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 361/734] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:41:15.079372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf8312-16ba-4ca4-a608-d86f0482d722 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:15.083298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:15.105890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedff40b-9321-447d-b531-f1f1393e5716 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:15.107411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedff40b-9321-447d-b531-f1f1393e5716 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:15.197188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf8312-16ba-4ca4-a608-d86f0482d722 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:15.226507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 374/735] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:41:15.231034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedff40b-9321-447d-b531-f1f1393e5716 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:15.238921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8947f-250b-4d6a-97df-be77d8093f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:15.239661 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8947f-250b-4d6a-97df-be77d8093f83 None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:15.241392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8947f-250b-4d6a-97df-be77d8093f83 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:15.241711 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8947f-250b-4d6a-97df-be77d8093f83 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:15.242450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8947f-250b-4d6a-97df-be77d8093f83 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:15.243843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8947f-250b-4d6a-97df-be77d8093f83 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:15.262330 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 362/736] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:41:15 2022] POST /identity/v3/services => generated 251 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:15.271882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8947f-250b-4d6a-97df-be77d8093f83 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:15.310573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:15.351862 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedff40b-9321-447d-b531-f1f1393e5716 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:15.355408 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:15.398562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc142add-7593-43a4-a990-9fcf8b7a8efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:15.399288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc142add-7593-43a4-a990-9fcf8b7a8efb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:15.399574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc142add-7593-43a4-a990-9fcf8b7a8efb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:15.399857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc142add-7593-43a4-a990-9fcf8b7a8efb None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:15.400625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc142add-7593-43a4-a990-9fcf8b7a8efb None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:15.400936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc142add-7593-43a4-a990-9fcf8b7a8efb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:15.402356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc142add-7593-43a4-a990-9fcf8b7a8efb None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:15.416565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8947f-250b-4d6a-97df-be77d8093f83 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:15.417132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 375/737] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:41:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:15.418155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc142add-7593-43a4-a990-9fcf8b7a8efb None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:15.458785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f042f-72fd-4348-b914-62d87e4f8319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:15.459638 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f042f-72fd-4348-b914-62d87e4f8319 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:15.459928 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f042f-72fd-4348-b914-62d87e4f8319 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:15.460210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f042f-72fd-4348-b914-62d87e4f8319 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:15.473018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f042f-72fd-4348-b914-62d87e4f8319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-379f042f-72fd-4348-b914-62d87e4f8319 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:15.480687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:15.481822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:15.482926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:15.482926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:15.482926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:15.482926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:15.482926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:15.482926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:15.482926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:15.482926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:15.482926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:15.482926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:15.482926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:15.482926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:15.482926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 363/738] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:41:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:15.483796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b94d22-7f28-407e-8b90-454456eb442c None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:15.483796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b94d22-7f28-407e-8b90-454456eb442c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:15.483796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b94d22-7f28-407e-8b90-454456eb442c None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:15.485294 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 364/739] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:15.485948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b94d22-7f28-407e-8b90-454456eb442c None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:15.538361 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc142add-7593-43a4-a990-9fcf8b7a8efb None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:15.563746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:15.579824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253c316-82c4-4b64-bc52-d40c7ae11d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:15.580624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253c316-82c4-4b64-bc52-d40c7ae11d0b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:15.580908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253c316-82c4-4b64-bc52-d40c7ae11d0b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:15.581830 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253c316-82c4-4b64-bc52-d40c7ae11d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:15.593693 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253c316-82c4-4b64-bc52-d40c7ae11d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:15.594637 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b94d22-7f28-407e-8b90-454456eb442c None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:15.595273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253c316-82c4-4b64-bc52-d40c7ae11d0b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-3253c316-82c4-4b64-bc52-d40c7ae11d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:15.614488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:15.615481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:15.616394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:15.616394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:15.616394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:15.616394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:15.616394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 376/740] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:15.616695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253c316-82c4-4b64-bc52-d40c7ae11d0b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:15.622210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:15.635054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982565d-2a7d-4182-9641-81f79ebbb8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:15.635930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982565d-2a7d-4182-9641-81f79ebbb8e8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:15.636220 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982565d-2a7d-4182-9641-81f79ebbb8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:15.636498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982565d-2a7d-4182-9641-81f79ebbb8e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:15.648508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982565d-2a7d-4182-9641-81f79ebbb8e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:15.649959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982565d-2a7d-4182-9641-81f79ebbb8e8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-4982565d-2a7d-4182-9641-81f79ebbb8e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:15.667417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:15.668336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:15.669211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:15.669211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:15.669211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:15.669211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:15.669211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 365/741] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:15.670190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da01767-2a3c-4b7c-aa1f-ee343353d984 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:15.670480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da01767-2a3c-4b7c-aa1f-ee343353d984 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:15.670758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da01767-2a3c-4b7c-aa1f-ee343353d984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:15.689409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6da01767-2a3c-4b7c-aa1f-ee343353d984 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:15.691380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6da01767-2a3c-4b7c-aa1f-ee343353d984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:15.730590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253c316-82c4-4b64-bc52-d40c7ae11d0b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:15.761142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 366/742] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:41:15.761984 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da01767-2a3c-4b7c-aa1f-ee343353d984 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:15.776321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878b75e-696c-4087-8cef-a86d340fb27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:15.777832 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878b75e-696c-4087-8cef-a86d340fb27f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:15.778587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878b75e-696c-4087-8cef-a86d340fb27f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:15.779273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878b75e-696c-4087-8cef-a86d340fb27f None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:15.780906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878b75e-696c-4087-8cef-a86d340fb27f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:15.781455 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878b75e-696c-4087-8cef-a86d340fb27f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:15.783123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878b75e-696c-4087-8cef-a86d340fb27f None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:15.795498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 377/743] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:41:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:15.796826 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878b75e-696c-4087-8cef-a86d340fb27f None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:15.842821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:15.873805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da01767-2a3c-4b7c-aa1f-ee343353d984 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:15.876869 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:15.907244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878b75e-696c-4087-8cef-a86d340fb27f None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:15.913372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342a6f3-707f-4959-a88f-73649c448672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:15.914443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342a6f3-707f-4959-a88f-73649c448672 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:15.914985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342a6f3-707f-4959-a88f-73649c448672 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:15.915471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342a6f3-707f-4959-a88f-73649c448672 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:15.916521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342a6f3-707f-4959-a88f-73649c448672 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:15.917033 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342a6f3-707f-4959-a88f-73649c448672 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:15.918680 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342a6f3-707f-4959-a88f-73649c448672 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:15.933125 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 367/744] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:15.934118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342a6f3-707f-4959-a88f-73649c448672 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:15.946089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e244e-96c2-407c-b3eb-af741d2116cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:15.946750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e244e-96c2-407c-b3eb-af741d2116cd None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:15.947457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e244e-96c2-407c-b3eb-af741d2116cd None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:15.947744 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e244e-96c2-407c-b3eb-af741d2116cd None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:15.948468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e244e-96c2-407c-b3eb-af741d2116cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:15.948775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e244e-96c2-407c-b3eb-af741d2116cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:15.950839 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e244e-96c2-407c-b3eb-af741d2116cd None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:15.963608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 378/745] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:15.964438 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e244e-96c2-407c-b3eb-af741d2116cd None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:16.053875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342a6f3-707f-4959-a88f-73649c448672 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:16.069191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:16.073400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e244e-96c2-407c-b3eb-af741d2116cd None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:16.091538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71700a9f-0f41-4dee-bbdc-6c92391fc290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:16.092210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71700a9f-0f41-4dee-bbdc-6c92391fc290 None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:16.092498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71700a9f-0f41-4dee-bbdc-6c92391fc290 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:16.092781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71700a9f-0f41-4dee-bbdc-6c92391fc290 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:16.093624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71700a9f-0f41-4dee-bbdc-6c92391fc290 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:16.095048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71700a9f-0f41-4dee-bbdc-6c92391fc290 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:16.109821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170617be-59f2-415d-95bb-f64665d7cb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:16.119162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 368/746] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:41:15 2022] POST /identity/v3/services => generated 240 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:16.119375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170617be-59f2-415d-95bb-f64665d7cb59 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:16.119375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170617be-59f2-415d-95bb-f64665d7cb59 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:16.119586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71700a9f-0f41-4dee-bbdc-6c92391fc290 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:16.119675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170617be-59f2-415d-95bb-f64665d7cb59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:16.132362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170617be-59f2-415d-95bb-f64665d7cb59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:16.133861 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170617be-59f2-415d-95bb-f64665d7cb59 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:16.149866 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-170617be-59f2-415d-95bb-f64665d7cb59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:16.154909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:16.155989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:16.157142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:16.157142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:16.157142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:16.157142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:16.157142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 379/747] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:16.157617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170617be-59f2-415d-95bb-f64665d7cb59 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:16.265757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71700a9f-0f41-4dee-bbdc-6c92391fc290 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:16.298137 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170617be-59f2-415d-95bb-f64665d7cb59 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:16.307648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ae1c3-4c38-4843-9f54-46864a5bc2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:16.308974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ae1c3-4c38-4843-9f54-46864a5bc2dd None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:16.309568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ae1c3-4c38-4843-9f54-46864a5bc2dd None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:16.310716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ae1c3-4c38-4843-9f54-46864a5bc2dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:16.320559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ae1c3-4c38-4843-9f54-46864a5bc2dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:16.320722 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ae1c3-4c38-4843-9f54-46864a5bc2dd None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-e54ae1c3-4c38-4843-9f54-46864a5bc2dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:16.337771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:16.338697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:16.339620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:16.339620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:16.339620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:16.339620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:16.339620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:16.339620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:16.339620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:16.339620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:16.340053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 369/748] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:16.345257 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ae1c3-4c38-4843-9f54-46864a5bc2dd None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:16.345407 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84730a8-863f-4bb1-ab3d-c169fa831848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:16.345407 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84730a8-863f-4bb1-ab3d-c169fa831848 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:16.345407 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84730a8-863f-4bb1-ab3d-c169fa831848 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:16.345407 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84730a8-863f-4bb1-ab3d-c169fa831848 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:16.353429 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84730a8-863f-4bb1-ab3d-c169fa831848 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:16.354890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84730a8-863f-4bb1-ab3d-c169fa831848 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-f84730a8-863f-4bb1-ab3d-c169fa831848 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:16.372540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:16.373532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:16.374472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:16.374472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:16.374472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:16.374472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:16.374472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:16.374472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:16.374472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:16.374472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:16.374472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:41:16.374472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:16.378915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 380/749] 10.222.0.52 () {56 vars in 1117 bytes} [Thu May 19 13:41:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:16.380293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84730a8-863f-4bb1-ab3d-c169fa831848 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:16.400932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:16.458825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:16.485600 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ae1c3-4c38-4843-9f54-46864a5bc2dd None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:16.506037 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84730a8-863f-4bb1-ab3d-c169fa831848 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:16.535146 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cc235-7d7d-473a-ac1a-8a55b0be278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:16.535811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cc235-7d7d-473a-ac1a-8a55b0be278a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:16.536088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cc235-7d7d-473a-ac1a-8a55b0be278a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:16.536356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cc235-7d7d-473a-ac1a-8a55b0be278a None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:16.537071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cc235-7d7d-473a-ac1a-8a55b0be278a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:16.537378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cc235-7d7d-473a-ac1a-8a55b0be278a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:16.538738 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cc235-7d7d-473a-ac1a-8a55b0be278a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:16.547016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1604730-f914-429d-945f-6335027b72fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:16.547781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1604730-f914-429d-945f-6335027b72fb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:16.548114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1604730-f914-429d-945f-6335027b72fb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:16.548902 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1604730-f914-429d-945f-6335027b72fb None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:16.550039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1604730-f914-429d-945f-6335027b72fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:16.550352 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1604730-f914-429d-945f-6335027b72fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:16.551872 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1604730-f914-429d-945f-6335027b72fb None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:16.552099 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 370/750] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:16.552948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cc235-7d7d-473a-ac1a-8a55b0be278a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:16.568442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 381/751] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:16.571707 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1604730-f914-429d-945f-6335027b72fb None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:16.660992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:16.691719 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cc235-7d7d-473a-ac1a-8a55b0be278a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:16.705368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1604730-f914-429d-945f-6335027b72fb None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:16.736992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bb951-47ab-43d6-9efb-b0108fcae506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:16.738737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bb951-47ab-43d6-9efb-b0108fcae506 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:16.739325 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bb951-47ab-43d6-9efb-b0108fcae506 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:16.739612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bb951-47ab-43d6-9efb-b0108fcae506 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:16.740892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bb951-47ab-43d6-9efb-b0108fcae506 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:16.741469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bb951-47ab-43d6-9efb-b0108fcae506 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:16.742923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bb951-47ab-43d6-9efb-b0108fcae506 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:16.745241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457cfba-caa9-4d3d-81d3-5d838b0464e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:16.746128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457cfba-caa9-4d3d-81d3-5d838b0464e6 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:16.746505 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457cfba-caa9-4d3d-81d3-5d838b0464e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:16.747149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457cfba-caa9-4d3d-81d3-5d838b0464e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:16.754678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:16.755967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9457cfba-caa9-4d3d-81d3-5d838b0464e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:16.757599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9457cfba-caa9-4d3d-81d3-5d838b0464e6 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:16.767430 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 371/752] 10.222.0.52 () {56 vars in 1149 bytes} [Thu May 19 13:41:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:16.768357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bb951-47ab-43d6-9efb-b0108fcae506 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-9457cfba-caa9-4d3d-81d3-5d838b0464e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:16.774823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:16.776058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:16.778077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:16.778077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:16.778077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:16.778077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:16.778077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:16.778077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:16.778077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:16.778077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:16.778077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:41:16.778077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:16.779358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 382/753] 10.222.0.52 () {56 vars in 1093 bytes} [Thu May 19 13:41:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:16.779358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457cfba-caa9-4d3d-81d3-5d838b0464e6 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:16.899070 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bb951-47ab-43d6-9efb-b0108fcae506 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:16.899998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457cfba-caa9-4d3d-81d3-5d838b0464e6 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:16.935562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde18c44-6ab0-49ef-a5be-baa6932d2d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:16.936138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde18c44-6ab0-49ef-a5be-baa6932d2d5b None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:16.936420 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde18c44-6ab0-49ef-a5be-baa6932d2d5b None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:16.936693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde18c44-6ab0-49ef-a5be-baa6932d2d5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:16.937514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde18c44-6ab0-49ef-a5be-baa6932d2d5b None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=8554ff40bbcb4fbb801d854d6527a6d2) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:16.937828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde18c44-6ab0-49ef-a5be-baa6932d2d5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:16.938090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:16.939266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde18c44-6ab0-49ef-a5be-baa6932d2d5b None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:16.946074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 372/754] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:16 2022] GET /identity/v3/role_assignments?user.id=8554ff40bbcb4fbb801d854d6527a6d2&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=0893bb944dd94634accd3906aa391cd2 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:16.946900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde18c44-6ab0-49ef-a5be-baa6932d2d5b None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:16.951788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49ee80-b942-4f0e-9815-628afd4f7eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:16.952466 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49ee80-b942-4f0e-9815-628afd4f7eb0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:16.952745 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49ee80-b942-4f0e-9815-628afd4f7eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:16.953016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49ee80-b942-4f0e-9815-628afd4f7eb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:17.072329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:17.099110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49ee80-b942-4f0e-9815-628afd4f7eb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:17.100654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49ee80-b942-4f0e-9815-628afd4f7eb0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:17.153815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde18c44-6ab0-49ef-a5be-baa6932d2d5b None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:17.230235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0dd4f-07fe-4713-a973-908e1981f569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:17.231263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0dd4f-07fe-4713-a973-908e1981f569 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:17.231552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0dd4f-07fe-4713-a973-908e1981f569 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:17.231825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0dd4f-07fe-4713-a973-908e1981f569 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:17.233386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0dd4f-07fe-4713-a973-908e1981f569 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:17.233701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0dd4f-07fe-4713-a973-908e1981f569 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:17.235968 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0dd4f-07fe-4713-a973-908e1981f569 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:17.257968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 383/755] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:41:17.260960 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 373/756] 10.222.0.52 () {56 vars in 1133 bytes} [Thu May 19 13:41:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:17.265285 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49ee80-b942-4f0e-9815-628afd4f7eb0 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:17.267930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0dd4f-07fe-4713-a973-908e1981f569 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:17.331607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:17.425782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0dd4f-07fe-4713-a973-908e1981f569 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:17.450378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:17.466393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d921a-8749-4b68-8f7b-a6f3a489cd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:17.467227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d921a-8749-4b68-8f7b-a6f3a489cd90 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:17.467507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d921a-8749-4b68-8f7b-a6f3a489cd90 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:17.467780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d921a-8749-4b68-8f7b-a6f3a489cd90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:17.479530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d921a-8749-4b68-8f7b-a6f3a489cd90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:17.481451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d921a-8749-4b68-8f7b-a6f3a489cd90 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-432d921a-8749-4b68-8f7b-a6f3a489cd90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:17.512739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:17.514929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:17.517039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:17.517039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:17.517039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:17.517039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:17.517039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 374/757] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:17.522888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d921a-8749-4b68-8f7b-a6f3a489cd90 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:17.524977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49ee80-b942-4f0e-9815-628afd4f7eb0 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:17.613465 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07da12-b479-4701-9160-81904fb1187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:17.614763 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07da12-b479-4701-9160-81904fb1187e None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:17.615289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07da12-b479-4701-9160-81904fb1187e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:17.615771 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07da12-b479-4701-9160-81904fb1187e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:17.628364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07da12-b479-4701-9160-81904fb1187e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:17.630289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07da12-b479-4701-9160-81904fb1187e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:17.631787 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d921a-8749-4b68-8f7b-a6f3a489cd90 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-be07da12-b479-4701-9160-81904fb1187e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:17.649705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:17.650652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:17.651586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:17.651586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:17.651586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:17.651586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:17.653900 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 384/758] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:17.655737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7668bb7-0f42-4b90-ab90-a774130a5b98 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:17.656241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7668bb7-0f42-4b90-ab90-a774130a5b98 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:17.656720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7668bb7-0f42-4b90-ab90-a774130a5b98 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:17.663127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 385/759] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:17.664287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7668bb7-0f42-4b90-ab90-a774130a5b98 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:17.672266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091f520-066b-4573-8f17-5b956d599dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:17.672941 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f520-066b-4573-8f17-5b956d599dba None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:17.673228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f520-066b-4573-8f17-5b956d599dba None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:17.673507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f520-066b-4573-8f17-5b956d599dba None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:17.674296 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091f520-066b-4573-8f17-5b956d599dba None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:17.674611 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091f520-066b-4573-8f17-5b956d599dba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:17.677096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091f520-066b-4573-8f17-5b956d599dba None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:17.696237 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 375/760] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:17.697594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4177c-968f-4474-a57d-307dbe4f20bf None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:17.697897 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4177c-968f-4474-a57d-307dbe4f20bf None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:17.698185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4177c-968f-4474-a57d-307dbe4f20bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:17.715157 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-d2c4177c-968f-4474-a57d-307dbe4f20bf None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:17.716360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d2c4177c-968f-4474-a57d-307dbe4f20bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:17.743824 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:17.763514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:17.785711 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7668bb7-0f42-4b90-ab90-a774130a5b98 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:17.794976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 376/761] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:17.795875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4177c-968f-4474-a57d-307dbe4f20bf None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:17.836920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf97ec-e6dc-4adb-823f-26eec579e4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:17.837736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf97ec-e6dc-4adb-823f-26eec579e4dd None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:17.838158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf97ec-e6dc-4adb-823f-26eec579e4dd None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:17.838453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf97ec-e6dc-4adb-823f-26eec579e4dd None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:17.839244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf97ec-e6dc-4adb-823f-26eec579e4dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:17.839557 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf97ec-e6dc-4adb-823f-26eec579e4dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:17.841737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf97ec-e6dc-4adb-823f-26eec579e4dd None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:17.858400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 386/762] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:17.859817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf97ec-e6dc-4adb-823f-26eec579e4dd None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:17.939113 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4177c-968f-4474-a57d-307dbe4f20bf None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:17.973247 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf97ec-e6dc-4adb-823f-26eec579e4dd None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:17.986590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7579ea1-260b-47e9-b826-e10c3d117863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:17.988791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7579ea1-260b-47e9-b826-e10c3d117863 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:17.989806 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7579ea1-260b-47e9-b826-e10c3d117863 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:17.990568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7579ea1-260b-47e9-b826-e10c3d117863 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:18.001276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7579ea1-260b-47e9-b826-e10c3d117863 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:18.003810 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7579ea1-260b-47e9-b826-e10c3d117863 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:18.013671 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018084e-1f2a-47d6-8189-3bbeef7fd897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:18.014484 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018084e-1f2a-47d6-8189-3bbeef7fd897 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:18.014763 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018084e-1f2a-47d6-8189-3bbeef7fd897 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:18.015071 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018084e-1f2a-47d6-8189-3bbeef7fd897 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:18.015729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018084e-1f2a-47d6-8189-3bbeef7fd897 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:18.017093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018084e-1f2a-47d6-8189-3bbeef7fd897 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-e7579ea1-260b-47e9-b826-e10c3d117863 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:18.022602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:18.023548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:18.024453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:18.024453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:18.024453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:18.024453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:18.024453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:18.024453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:18.024453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:18.024453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:18.024453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 377/763] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:18.024943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7579ea1-260b-47e9-b826-e10c3d117863 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-e018084e-1f2a-47d6-8189-3bbeef7fd897 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:18.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:18.036952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:18.037802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 387/764] 10.222.0.52 () {56 vars in 1129 bytes} [Thu May 19 13:41:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:18.038677 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018084e-1f2a-47d6-8189-3bbeef7fd897 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:18.049127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:18.069094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:18.159747 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7579ea1-260b-47e9-b826-e10c3d117863 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:18.175477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018084e-1f2a-47d6-8189-3bbeef7fd897 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:18.207851 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6c3f0-60fd-4138-bf21-9e0bc7734abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:18.209188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6c3f0-60fd-4138-bf21-9e0bc7734abe None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:18.209717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6c3f0-60fd-4138-bf21-9e0bc7734abe None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:18.210181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6c3f0-60fd-4138-bf21-9e0bc7734abe None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:18.211646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9ed44-3566-4eb0-8e93-ee53cb27c44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:18.212397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9ed44-3566-4eb0-8e93-ee53cb27c44e None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:18.212830 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9ed44-3566-4eb0-8e93-ee53cb27c44e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:18.213066 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9ed44-3566-4eb0-8e93-ee53cb27c44e None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:18.213920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9ed44-3566-4eb0-8e93-ee53cb27c44e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:18.214403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9ed44-3566-4eb0-8e93-ee53cb27c44e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:18.215836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9ed44-3566-4eb0-8e93-ee53cb27c44e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:18.219708 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6c3f0-60fd-4138-bf21-9e0bc7734abe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:18.221336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6c3f0-60fd-4138-bf21-9e0bc7734abe None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:18.229252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 388/765] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:18.230175 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9ed44-3566-4eb0-8e93-ee53cb27c44e None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-d8f6c3f0-60fd-4138-bf21-9e0bc7734abe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:18.241707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:18.242712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:18.243642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:18.243642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:18.243642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:18.243642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:18.243642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:18.243642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:18.243642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:18.243642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:18.244861 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 378/766] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:18.246124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6c3f0-60fd-4138-bf21-9e0bc7734abe None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:18.305252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:18.348386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9ed44-3566-4eb0-8e93-ee53cb27c44e None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:18.369342 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:18.390584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a6771-8ab3-4749-94fc-ffbb8da4cca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:18.391513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a6771-8ab3-4749-94fc-ffbb8da4cca9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:18.391626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a6771-8ab3-4749-94fc-ffbb8da4cca9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:18.391817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a6771-8ab3-4749-94fc-ffbb8da4cca9 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:18.392532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a6771-8ab3-4749-94fc-ffbb8da4cca9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:18.392845 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a6771-8ab3-4749-94fc-ffbb8da4cca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:18.395737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a6771-8ab3-4749-94fc-ffbb8da4cca9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:18.400856 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6c3f0-60fd-4138-bf21-9e0bc7734abe None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:18.402821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 389/767] 10.222.0.52 () {56 vars in 1124 bytes} [Thu May 19 13:41:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:18.403749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a6771-8ab3-4749-94fc-ffbb8da4cca9 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:18.443371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ea7e3-f35e-4b59-82df-43771f6f5d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:18.444981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ea7e3-f35e-4b59-82df-43771f6f5d2d None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:18.446144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ea7e3-f35e-4b59-82df-43771f6f5d2d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:18.446144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ea7e3-f35e-4b59-82df-43771f6f5d2d None admin] PATH_INFO: `/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/4f6093d0c1fc41d98c58bace81e3d8e5/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:18.448573 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ea7e3-f35e-4b59-82df-43771f6f5d2d None admin] RBAC: Authorizing `identity:create_grant(project_id=2fc6fd42a31942688f570f1ce86be510, user_id=4f6093d0c1fc41d98c58bace81e3d8e5, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:18.450073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ea7e3-f35e-4b59-82df-43771f6f5d2d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:18.466835 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 379/768] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:41:18 2022] PUT /identity/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/4f6093d0c1fc41d98c58bace81e3d8e5/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:41:18.467943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ea7e3-f35e-4b59-82df-43771f6f5d2d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:18.524187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a6771-8ab3-4749-94fc-ffbb8da4cca9 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:18.575729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:18.582748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3ca39-8b94-495d-a044-c719ab37e5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:18.584667 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3ca39-8b94-495d-a044-c719ab37e5d2 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:18.586545 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3ca39-8b94-495d-a044-c719ab37e5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:18.587055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3ca39-8b94-495d-a044-c719ab37e5d2 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:18.588247 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3ca39-8b94-495d-a044-c719ab37e5d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:18.590159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3ca39-8b94-495d-a044-c719ab37e5d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:18.591967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3ca39-8b94-495d-a044-c719ab37e5d2 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:18.609362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 390/769] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:18.610891 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2367c-94d6-4795-a4af-c533cb2b0d63 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:18.611200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2367c-94d6-4795-a4af-c533cb2b0d63 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:18.611475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2367c-94d6-4795-a4af-c533cb2b0d63 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:18.613587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 391/770] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:18.614935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2367c-94d6-4795-a4af-c533cb2b0d63 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:18.617025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ea7e3-f35e-4b59-82df-43771f6f5d2d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:18.669837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced83a1-cd51-44a7-a773-8b4b2f116167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:18.671233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced83a1-cd51-44a7-a773-8b4b2f116167 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:18.671700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced83a1-cd51-44a7-a773-8b4b2f116167 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:18.672169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced83a1-cd51-44a7-a773-8b4b2f116167 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:18.673848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced83a1-cd51-44a7-a773-8b4b2f116167 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:18.674383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced83a1-cd51-44a7-a773-8b4b2f116167 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:18.677083 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced83a1-cd51-44a7-a773-8b4b2f116167 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:18.688430 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 380/771] 10.222.0.52 () {56 vars in 1124 bytes} [Thu May 19 13:41:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:18.689733 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676503ff-de5c-434b-a661-156f49147c30 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:18.690032 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676503ff-de5c-434b-a661-156f49147c30 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:18.690893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676503ff-de5c-434b-a661-156f49147c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:18.704503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:18.712148 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-676503ff-de5c-434b-a661-156f49147c30 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:18.713371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-676503ff-de5c-434b-a661-156f49147c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:18.743952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2367c-94d6-4795-a4af-c533cb2b0d63 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:18.793939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f5adc-ec6e-45e0-ae43-1d648a9deee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:18.794879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f5adc-ec6e-45e0-ae43-1d648a9deee2 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:18.795596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f5adc-ec6e-45e0-ae43-1d648a9deee2 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:18.795885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f5adc-ec6e-45e0-ae43-1d648a9deee2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:18.797018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f5adc-ec6e-45e0-ae43-1d648a9deee2 None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=a989e5551c1640ebbde8768247441627) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:18.797598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f5adc-ec6e-45e0-ae43-1d648a9deee2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:18.800321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f5adc-ec6e-45e0-ae43-1d648a9deee2 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:18.803735 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 381/772] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:18.804599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676503ff-de5c-434b-a661-156f49147c30 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:18.811114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 392/773] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:18 2022] GET /identity/v3/role_assignments?user.id=a989e5551c1640ebbde8768247441627&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:18.836113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f5adc-ec6e-45e0-ae43-1d648a9deee2 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:18.933597 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676503ff-de5c-434b-a661-156f49147c30 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:18.952972 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:18.974426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359695ff-f932-44ef-a0b6-48b4c886cd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:18.975135 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359695ff-f932-44ef-a0b6-48b4c886cd0c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:18.975425 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359695ff-f932-44ef-a0b6-48b4c886cd0c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:18.975790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359695ff-f932-44ef-a0b6-48b4c886cd0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:18.976513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359695ff-f932-44ef-a0b6-48b4c886cd0c None admin] RBAC: Adding query filter params (interface=public, service_id=ee3ff85a3607438daf080460d80c0031, region_id=RegionOne) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:18.976831 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359695ff-f932-44ef-a0b6-48b4c886cd0c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:18.978236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359695ff-f932-44ef-a0b6-48b4c886cd0c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:18.983589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:18.993774 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 382/774] 10.222.0.52 () {56 vars in 1251 bytes} [Thu May 19 13:41:18 2022] GET /identity/v3/endpoints?service_id=ee3ff85a3607438daf080460d80c0031&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:18.997962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359695ff-f932-44ef-a0b6-48b4c886cd0c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:19.020112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f5adc-ec6e-45e0-ae43-1d648a9deee2 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:19.075630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc358e1-67fd-4a50-9f54-6d9a2b015731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:19.076440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc358e1-67fd-4a50-9f54-6d9a2b015731 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:19.076725 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc358e1-67fd-4a50-9f54-6d9a2b015731 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:19.077002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc358e1-67fd-4a50-9f54-6d9a2b015731 None admin] PATH_INFO: `/v3/services/network` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:19.077671 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc358e1-67fd-4a50-9f54-6d9a2b015731 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:19.079069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc358e1-67fd-4a50-9f54-6d9a2b015731 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-1dc358e1-67fd-4a50-9f54-6d9a2b015731 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:19.101935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:19.102977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:41:19.103873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:19.113992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 393/775] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:18 2022] GET /identity/v3/services/network => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:19.120129 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc358e1-67fd-4a50-9f54-6d9a2b015731 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:19.137988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359695ff-f932-44ef-a0b6-48b4c886cd0c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:19.178338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725afde-a8ea-4275-bfdd-316ba371e2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:19.179044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725afde-a8ea-4275-bfdd-316ba371e2b9 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:19.179335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725afde-a8ea-4275-bfdd-316ba371e2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:19.179619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725afde-a8ea-4275-bfdd-316ba371e2b9 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:19.180313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725afde-a8ea-4275-bfdd-316ba371e2b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:19.181688 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725afde-a8ea-4275-bfdd-316ba371e2b9 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:19.192689 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 383/776] 10.222.0.52 () {56 vars in 1087 bytes} [Thu May 19 13:41:18 2022] GET /identity/v3/services => generated 772 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:19.302127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc358e1-67fd-4a50-9f54-6d9a2b015731 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:19.332275 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:19.355880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae46e4-966a-4b93-ba40-9ed414a944cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:19.356485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae46e4-966a-4b93-ba40-9ed414a944cb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:19.356787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae46e4-966a-4b93-ba40-9ed414a944cb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:19.357052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae46e4-966a-4b93-ba40-9ed414a944cb None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:19.357781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae46e4-966a-4b93-ba40-9ed414a944cb None admin] RBAC: Adding query filter params (name=network) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:19.358091 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae46e4-966a-4b93-ba40-9ed414a944cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:19.359529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae46e4-966a-4b93-ba40-9ed414a944cb None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:19.371512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 394/777] 10.222.0.52 () {56 vars in 1112 bytes} [Thu May 19 13:41:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:19.377114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725afde-a8ea-4275-bfdd-316ba371e2b9 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:19.454022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:19.495373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725afde-a8ea-4275-bfdd-316ba371e2b9 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:19.534057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614184a-8d7d-4bab-bd0c-7d1b11c375b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:19.536184 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614184a-8d7d-4bab-bd0c-7d1b11c375b3 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:19.536668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614184a-8d7d-4bab-bd0c-7d1b11c375b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:19.537162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614184a-8d7d-4bab-bd0c-7d1b11c375b3 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:19.538086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614184a-8d7d-4bab-bd0c-7d1b11c375b3 None admin] RBAC: Adding query filter params (type=network) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:19.538629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614184a-8d7d-4bab-bd0c-7d1b11c375b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:19.540276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614184a-8d7d-4bab-bd0c-7d1b11c375b3 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:19.547857 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 384/778] 10.222.0.52 () {56 vars in 1112 bytes} [Thu May 19 13:41:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:19.554607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae46e4-966a-4b93-ba40-9ed414a944cb None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:19.715385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae46e4-966a-4b93-ba40-9ed414a944cb None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:19.768741 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c02cd9-7a2b-4749-bac2-f8e21aba2ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:19.769560 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c02cd9-7a2b-4749-bac2-f8e21aba2ea3 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:19.769853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c02cd9-7a2b-4749-bac2-f8e21aba2ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:19.770131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c02cd9-7a2b-4749-bac2-f8e21aba2ea3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:19.770954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c02cd9-7a2b-4749-bac2-f8e21aba2ea3 None admin] RBAC: Adding query filter params (interface=public, service_id=7b3f9dbe845f43ffa73859b50c82ff77, region_id=RegionOne) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:19.771276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c02cd9-7a2b-4749-bac2-f8e21aba2ea3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:19.772651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c02cd9-7a2b-4749-bac2-f8e21aba2ea3 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:19.785457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 395/779] 10.222.0.52 () {56 vars in 1251 bytes} [Thu May 19 13:41:19 2022] GET /identity/v3/endpoints?service_id=7b3f9dbe845f43ffa73859b50c82ff77&interface=public®ion_id=RegionOne => generated 194 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:19.790455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614184a-8d7d-4bab-bd0c-7d1b11c375b3 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:19.823901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543601e-5aa4-434e-a000-92ddbbacfead None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:19.824244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543601e-5aa4-434e-a000-92ddbbacfead None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:19.824467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543601e-5aa4-434e-a000-92ddbbacfead None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:19.826296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 396/780] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:41:19.835548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72700f-fa57-4f02-9eaa-e2950da9d5f6 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:19.835838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72700f-fa57-4f02-9eaa-e2950da9d5f6 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:19.836122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72700f-fa57-4f02-9eaa-e2950da9d5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:19.864754 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-da72700f-fa57-4f02-9eaa-e2950da9d5f6 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:19.865970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-da72700f-fa57-4f02-9eaa-e2950da9d5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:19.924140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:19.968014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614184a-8d7d-4bab-bd0c-7d1b11c375b3 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:19.997892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:20.010889 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 397/781] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:20.014790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628f337-ee26-415f-8715-707a1575b465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:20.015651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628f337-ee26-415f-8715-707a1575b465 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:20.016091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628f337-ee26-415f-8715-707a1575b465 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:20.016518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628f337-ee26-415f-8715-707a1575b465 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:20.017379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628f337-ee26-415f-8715-707a1575b465 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:20.020084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628f337-ee26-415f-8715-707a1575b465 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:20.029760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72700f-fa57-4f02-9eaa-e2950da9d5f6 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:20.032813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 385/782] 10.222.0.52 () {56 vars in 1087 bytes} [Thu May 19 13:41:19 2022] GET /identity/v3/services => generated 772 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:20.175461 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72700f-fa57-4f02-9eaa-e2950da9d5f6 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:20.226016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47a0dd-1c69-4c8d-84d5-346f583528dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:20.226748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47a0dd-1c69-4c8d-84d5-346f583528dc None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:20.227064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47a0dd-1c69-4c8d-84d5-346f583528dc None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:20.227344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47a0dd-1c69-4c8d-84d5-346f583528dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:20.240256 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47a0dd-1c69-4c8d-84d5-346f583528dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-6c47a0dd-1c69-4c8d-84d5-346f583528dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:20.248345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:20.249322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:20.250178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:20.250178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:20.250178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:20.250178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:20.250178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:20.250178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:20.250178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:20.250178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:20.250178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:20.250178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:20.250178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:20.250178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:20.250178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 398/783] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:41:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:20.256080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628f337-ee26-415f-8715-707a1575b465 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:20.387279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628f337-ee26-415f-8715-707a1575b465 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:20.425376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43861b61-1cf0-4d11-afaf-47606072b891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:20.426047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43861b61-1cf0-4d11-afaf-47606072b891 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:20.426329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43861b61-1cf0-4d11-afaf-47606072b891 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:20.426599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43861b61-1cf0-4d11-afaf-47606072b891 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:20.427367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43861b61-1cf0-4d11-afaf-47606072b891 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:20.427670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43861b61-1cf0-4d11-afaf-47606072b891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:20.429082 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43861b61-1cf0-4d11-afaf-47606072b891 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:20.437765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 386/784] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:41:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:20.443418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47a0dd-1c69-4c8d-84d5-346f583528dc None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:20.485321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:20.587084 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47a0dd-1c69-4c8d-84d5-346f583528dc None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:20.638372 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54716bd2-7ee5-432a-9284-8e2199a0515d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:20.639205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54716bd2-7ee5-432a-9284-8e2199a0515d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:20.639491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54716bd2-7ee5-432a-9284-8e2199a0515d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:20.639770 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54716bd2-7ee5-432a-9284-8e2199a0515d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:20.656072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54716bd2-7ee5-432a-9284-8e2199a0515d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:20.657534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54716bd2-7ee5-432a-9284-8e2199a0515d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-54716bd2-7ee5-432a-9284-8e2199a0515d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:20.683576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:20.684529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:20.685416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:20.685416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:20.685416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:20.685416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:20.686074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 399/785] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:20.695982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43861b61-1cf0-4d11-afaf-47606072b891 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:20.807348 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:20.906947 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43861b61-1cf0-4d11-afaf-47606072b891 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:20.963367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28a11c-4de3-4a79-8838-8395f1c682a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:20.964436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28a11c-4de3-4a79-8838-8395f1c682a3 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:20.964929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28a11c-4de3-4a79-8838-8395f1c682a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:20.965432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28a11c-4de3-4a79-8838-8395f1c682a3 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:20.966361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28a11c-4de3-4a79-8838-8395f1c682a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:20.966891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28a11c-4de3-4a79-8838-8395f1c682a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:20.968545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28a11c-4de3-4a79-8838-8395f1c682a3 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:20.985632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 387/786] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:20.991067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54716bd2-7ee5-432a-9284-8e2199a0515d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:21.117625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:21.133195 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54716bd2-7ee5-432a-9284-8e2199a0515d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:21.183829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1c9c0-af38-4f25-bcad-9cfab2abe55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:21.184676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1c9c0-af38-4f25-bcad-9cfab2abe55a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:21.184962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1c9c0-af38-4f25-bcad-9cfab2abe55a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:21.185355 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1c9c0-af38-4f25-bcad-9cfab2abe55a None admin] PATH_INFO: `/v3/users/swift` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:21.195776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1c9c0-af38-4f25-bcad-9cfab2abe55a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:21.197245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1c9c0-af38-4f25-bcad-9cfab2abe55a None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-27e1c9c0-af38-4f25-bcad-9cfab2abe55a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:21.218911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:21.220425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:21.221748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:21.221748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:21.221748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:21.221748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:21.221748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:21.221748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:21.221748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:21.221748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:21.221748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 13:41:21.221748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:21.221748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 400/787] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:41:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:21.226037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28a11c-4de3-4a79-8838-8395f1c682a3 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:21.390971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bab9a-89d3-44e1-996a-07b233da1a27 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:21.390971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bab9a-89d3-44e1-996a-07b233da1a27 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:21.390971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bab9a-89d3-44e1-996a-07b233da1a27 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:21.392076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 401/788] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:21.397641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70460da-d99c-4ce0-a532-a1508722e22a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:21.397820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70460da-d99c-4ce0-a532-a1508722e22a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:21.398048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70460da-d99c-4ce0-a532-a1508722e22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:21.427941 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-b70460da-d99c-4ce0-a532-a1508722e22a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:21.429158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-b70460da-d99c-4ce0-a532-a1508722e22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:21.436360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28a11c-4de3-4a79-8838-8395f1c682a3 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:21.463224 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:21.486206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0c254-8600-466c-993c-12443c55ec8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:21.487004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0c254-8600-466c-993c-12443c55ec8b None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:21.487859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0c254-8600-466c-993c-12443c55ec8b None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:21.488046 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0c254-8600-466c-993c-12443c55ec8b None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:21.489127 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0c254-8600-466c-993c-12443c55ec8b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:21.489441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0c254-8600-466c-993c-12443c55ec8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:21.490892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0c254-8600-466c-993c-12443c55ec8b None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:21.510381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 388/789] 10.222.0.52 () {56 vars in 1135 bytes} [Thu May 19 13:41:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:21.511891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2181534-6c12-41a4-972f-9b07780b6445 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:21.512190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2181534-6c12-41a4-972f-9b07780b6445 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:21.512474 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2181534-6c12-41a4-972f-9b07780b6445 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:21.513998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 389/790] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:21.515117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2181534-6c12-41a4-972f-9b07780b6445 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:21.539127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 402/791] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:21.540566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856dfa1-72a0-49c9-b23d-2609b907cfd8 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:21.540864 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856dfa1-72a0-49c9-b23d-2609b907cfd8 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:21.541145 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856dfa1-72a0-49c9-b23d-2609b907cfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:21.569903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-3856dfa1-72a0-49c9-b23d-2609b907cfd8 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:21.571131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3856dfa1-72a0-49c9-b23d-2609b907cfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:21.623126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2181534-6c12-41a4-972f-9b07780b6445 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:21.661892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7138700-1557-4fe1-ba83-300b0b174d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:21.662722 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7138700-1557-4fe1-ba83-300b0b174d4f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:21.663163 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 403/792] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:21.663304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7138700-1557-4fe1-ba83-300b0b174d4f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:21.663689 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7138700-1557-4fe1-ba83-300b0b174d4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:21.664373 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856dfa1-72a0-49c9-b23d-2609b907cfd8 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:21.675213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7138700-1557-4fe1-ba83-300b0b174d4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:21.676389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:21.677645 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7138700-1557-4fe1-ba83-300b0b174d4f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-d7138700-1557-4fe1-ba83-300b0b174d4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:21.697327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:21.698501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:21.700081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:21.700081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:21.700081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:21.700081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:21.701185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 390/793] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:21.706501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7138700-1557-4fe1-ba83-300b0b174d4f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:21.735140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:21.794796 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856dfa1-72a0-49c9-b23d-2609b907cfd8 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:21.822482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7138700-1557-4fe1-ba83-300b0b174d4f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:21.834368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68ce8e-07c3-441b-873a-9a7bc43b0d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:21.835502 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68ce8e-07c3-441b-873a-9a7bc43b0d6a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:21.835946 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68ce8e-07c3-441b-873a-9a7bc43b0d6a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:21.836381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68ce8e-07c3-441b-873a-9a7bc43b0d6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:21.848484 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68ce8e-07c3-441b-873a-9a7bc43b0d6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-3f68ce8e-07c3-441b-873a-9a7bc43b0d6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:21.856820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:21.857894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:21.858731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:21.858731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:21.858731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:21.858731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:21.858731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:21.858731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:21.858731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:21.858731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:21.858731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:21.858731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:21.858731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:21.858731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:21.862905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114607b-5c1a-45c9-b966-9ceeaf2a8778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:21.862905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114607b-5c1a-45c9-b966-9ceeaf2a8778 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:21.862905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114607b-5c1a-45c9-b966-9ceeaf2a8778 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:21.862905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114607b-5c1a-45c9-b966-9ceeaf2a8778 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:21.863365 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114607b-5c1a-45c9-b966-9ceeaf2a8778 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:21.863716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114607b-5c1a-45c9-b966-9ceeaf2a8778 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:21.865091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114607b-5c1a-45c9-b966-9ceeaf2a8778 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:21.866290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 404/794] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:41:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:21.867381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68ce8e-07c3-441b-873a-9a7bc43b0d6a None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:21.878854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 391/795] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:21.879977 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114607b-5c1a-45c9-b966-9ceeaf2a8778 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:21.938959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:21.974929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:21.975362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68ce8e-07c3-441b-873a-9a7bc43b0d6a None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:21.991595 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114607b-5c1a-45c9-b966-9ceeaf2a8778 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:22.013271 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252430a-8ae0-47d4-866d-f46db5a27c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:22.014069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252430a-8ae0-47d4-866d-f46db5a27c5d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:22.014359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252430a-8ae0-47d4-866d-f46db5a27c5d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:22.014641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252430a-8ae0-47d4-866d-f46db5a27c5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:22.026816 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252430a-8ae0-47d4-866d-f46db5a27c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:22.027205 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615f919-0a1d-4fba-8ba6-453e4cee39a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:22.027867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615f919-0a1d-4fba-8ba6-453e4cee39a7 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:22.028156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615f919-0a1d-4fba-8ba6-453e4cee39a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:22.028436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615f919-0a1d-4fba-8ba6-453e4cee39a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:22.029185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615f919-0a1d-4fba-8ba6-453e4cee39a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:22.029495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615f919-0a1d-4fba-8ba6-453e4cee39a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:22.030852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615f919-0a1d-4fba-8ba6-453e4cee39a7 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-7252430a-8ae0-47d4-866d-f46db5a27c5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:22.035646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:22.036595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:22.037440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:22.037440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:22.037440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:22.037440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:22.037440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:22.037440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:22.037440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:22.037440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:22.037440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:22.037440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:22.037440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:22.037440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:22.037440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 405/796] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:41:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:22.038142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252430a-8ae0-47d4-866d-f46db5a27c5d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:22.044695 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 392/797] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:41:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:22.044695 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44facf3b-cab0-4ea1-b90c-ae4dd454f469 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:22.044695 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44facf3b-cab0-4ea1-b90c-ae4dd454f469 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:22.044695 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44facf3b-cab0-4ea1-b90c-ae4dd454f469 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:22.044695 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 393/798] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:22.044695 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44facf3b-cab0-4ea1-b90c-ae4dd454f469 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:22.147776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252430a-8ae0-47d4-866d-f46db5a27c5d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:22.163917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44facf3b-cab0-4ea1-b90c-ae4dd454f469 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:22.174269 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:22.183992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90b4d1-aaeb-48b9-b003-babf3ec92879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:22.185131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90b4d1-aaeb-48b9-b003-babf3ec92879 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:22.185641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90b4d1-aaeb-48b9-b003-babf3ec92879 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:22.186145 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90b4d1-aaeb-48b9-b003-babf3ec92879 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:22.194536 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90b4d1-aaeb-48b9-b003-babf3ec92879 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:22.196432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90b4d1-aaeb-48b9-b003-babf3ec92879 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:22.202846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55f3bc-ef40-4e41-aae3-480d490dcdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:22.204156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55f3bc-ef40-4e41-aae3-480d490dcdbf None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:22.204642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55f3bc-ef40-4e41-aae3-480d490dcdbf None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:22.205135 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55f3bc-ef40-4e41-aae3-480d490dcdbf None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:22.206681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55f3bc-ef40-4e41-aae3-480d490dcdbf None admin] RBAC: Adding query filter params (name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:22.207304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55f3bc-ef40-4e41-aae3-480d490dcdbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:22.208875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55f3bc-ef40-4e41-aae3-480d490dcdbf None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-1b90b4d1-aaeb-48b9-b003-babf3ec92879 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:22.212188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:22.213064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:22.213982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:22.213982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:22.213982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:22.213982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:22.213982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:22.213982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:22.213982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:22.213982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:22.213982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 406/799] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:22.214901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90b4d1-aaeb-48b9-b003-babf3ec92879 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:22.222733 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 394/800] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:41:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:22.224412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01ff66-0d4d-4fb0-af74-3982712b3045 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:22.224857 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01ff66-0d4d-4fb0-af74-3982712b3045 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:22.225388 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01ff66-0d4d-4fb0-af74-3982712b3045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:22.243925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-7d01ff66-0d4d-4fb0-af74-3982712b3045 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:22.245113 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-7d01ff66-0d4d-4fb0-af74-3982712b3045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:22.247845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:22.319740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90b4d1-aaeb-48b9-b003-babf3ec92879 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:22.322506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 395/801] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:22.323369 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01ff66-0d4d-4fb0-af74-3982712b3045 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:22.369951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0ef09-8d29-4060-a195-887cd5be6341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:22.370846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0ef09-8d29-4060-a195-887cd5be6341 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:22.371166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0ef09-8d29-4060-a195-887cd5be6341 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:22.371446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0ef09-8d29-4060-a195-887cd5be6341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:22.383389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0ef09-8d29-4060-a195-887cd5be6341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:22.387573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0ef09-8d29-4060-a195-887cd5be6341 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-bea0ef09-8d29-4060-a195-887cd5be6341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:22.407192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:22.408072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:22.408969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:22.408969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:22.408969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:22.408969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:22.408969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 407/802] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:22.410069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0ef09-8d29-4060-a195-887cd5be6341 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:22.448007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01ff66-0d4d-4fb0-af74-3982712b3045 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:22.456513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:22.484557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c699dc3-32cb-4563-8c9e-f727f8f49b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:22.485223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c699dc3-32cb-4563-8c9e-f727f8f49b2f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:22.485506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c699dc3-32cb-4563-8c9e-f727f8f49b2f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:22.485780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c699dc3-32cb-4563-8c9e-f727f8f49b2f None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:22.486562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c699dc3-32cb-4563-8c9e-f727f8f49b2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:22.486887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c699dc3-32cb-4563-8c9e-f727f8f49b2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:22.489000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c699dc3-32cb-4563-8c9e-f727f8f49b2f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:22.497702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:22.508598 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 396/803] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:22.509486 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c699dc3-32cb-4563-8c9e-f727f8f49b2f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:22.524321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0ef09-8d29-4060-a195-887cd5be6341 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:22.564764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241d19f-ddf6-439a-b353-c57172c860fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:22.565738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241d19f-ddf6-439a-b353-c57172c860fa None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:22.566013 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241d19f-ddf6-439a-b353-c57172c860fa None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:22.566245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241d19f-ddf6-439a-b353-c57172c860fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:22.577587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241d19f-ddf6-439a-b353-c57172c860fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:22.579222 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241d19f-ddf6-439a-b353-c57172c860fa None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-5241d19f-ddf6-439a-b353-c57172c860fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:22.599782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:22.600781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:22.601669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:22.601669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:22.601669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:22.601669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:22.605951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 408/804] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:22.607205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241d19f-ddf6-439a-b353-c57172c860fa None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:22.617146 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c699dc3-32cb-4563-8c9e-f727f8f49b2f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:22.654684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1078b44-6b5e-4125-b76a-e6bb62a5c5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:22.655676 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078b44-6b5e-4125-b76a-e6bb62a5c5a5 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:22.655967 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078b44-6b5e-4125-b76a-e6bb62a5c5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:22.656242 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078b44-6b5e-4125-b76a-e6bb62a5c5a5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:22.656896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1078b44-6b5e-4125-b76a-e6bb62a5c5a5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:22.658256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1078b44-6b5e-4125-b76a-e6bb62a5c5a5 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-b1078b44-6b5e-4125-b76a-e6bb62a5c5a5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:22.671384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:22.672441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:41:22.673259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:22.677979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 397/805] 10.222.0.52 () {56 vars in 1129 bytes} [Thu May 19 13:41:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:22.678795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1078b44-6b5e-4125-b76a-e6bb62a5c5a5 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:22.712375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:22.723490 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241d19f-ddf6-439a-b353-c57172c860fa None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:22.744063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:22.762600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561c08d-0be4-4a38-9359-dd31baa3ad2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:22.763288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561c08d-0be4-4a38-9359-dd31baa3ad2d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:22.763570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561c08d-0be4-4a38-9359-dd31baa3ad2d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:22.763849 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561c08d-0be4-4a38-9359-dd31baa3ad2d None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:22.764571 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561c08d-0be4-4a38-9359-dd31baa3ad2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:22.764875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561c08d-0be4-4a38-9359-dd31baa3ad2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:22.766227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561c08d-0be4-4a38-9359-dd31baa3ad2d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:22.783626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 409/806] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:22.783626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561c08d-0be4-4a38-9359-dd31baa3ad2d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:22.794048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1078b44-6b5e-4125-b76a-e6bb62a5c5a5 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:22.827751 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b133d-e137-4e43-99ec-f52c49eb4d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:22.828626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b133d-e137-4e43-99ec-f52c49eb4d31 None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:22.828910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b133d-e137-4e43-99ec-f52c49eb4d31 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:22.829181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b133d-e137-4e43-99ec-f52c49eb4d31 None admin] PATH_INFO: `/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/8554ff40bbcb4fbb801d854d6527a6d2/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:22.831220 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b133d-e137-4e43-99ec-f52c49eb4d31 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fc6fd42a31942688f570f1ce86be510, user_id=8554ff40bbcb4fbb801d854d6527a6d2, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:22.832640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b133d-e137-4e43-99ec-f52c49eb4d31 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:22.845328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 398/807] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:41:22 2022] PUT /identity/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/8554ff40bbcb4fbb801d854d6527a6d2/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:41:22.846109 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b133d-e137-4e43-99ec-f52c49eb4d31 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:22.890953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561c08d-0be4-4a38-9359-dd31baa3ad2d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:22.933698 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225375f8-6995-425f-9948-f20531b600a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:22.934369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225375f8-6995-425f-9948-f20531b600a6 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:22.934651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225375f8-6995-425f-9948-f20531b600a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:22.934952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225375f8-6995-425f-9948-f20531b600a6 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:22.935821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225375f8-6995-425f-9948-f20531b600a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:22.936128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225375f8-6995-425f-9948-f20531b600a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:22.937643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225375f8-6995-425f-9948-f20531b600a6 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:22.956085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 410/808] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:22.956085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225375f8-6995-425f-9948-f20531b600a6 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:22.967616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b133d-e137-4e43-99ec-f52c49eb4d31 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:22.973297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:22.993717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:23.011801 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38b3bb-7df4-47bb-8000-ada324241eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:23.012489 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38b3bb-7df4-47bb-8000-ada324241eb7 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:23.012780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38b3bb-7df4-47bb-8000-ada324241eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:23.013064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38b3bb-7df4-47bb-8000-ada324241eb7 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:23.013791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38b3bb-7df4-47bb-8000-ada324241eb7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:23.014103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38b3bb-7df4-47bb-8000-ada324241eb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:23.015540 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38b3bb-7df4-47bb-8000-ada324241eb7 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:23.025154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 399/809] 10.222.0.52 () {56 vars in 1124 bytes} [Thu May 19 13:41:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:23.025154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54cf13-960c-4ec9-95b0-184728dfc3d7 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:23.025154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54cf13-960c-4ec9-95b0-184728dfc3d7 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:23.025154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54cf13-960c-4ec9-95b0-184728dfc3d7 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:23.025154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 400/810] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:23.025154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54cf13-960c-4ec9-95b0-184728dfc3d7 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:23.085279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225375f8-6995-425f-9948-f20531b600a6 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:23.122970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a6b7b-0be6-4c16-8c8c-e4bd6b173c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:23.123767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a6b7b-0be6-4c16-8c8c-e4bd6b173c06 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:23.124054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a6b7b-0be6-4c16-8c8c-e4bd6b173c06 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:23.124330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a6b7b-0be6-4c16-8c8c-e4bd6b173c06 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:23.132187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a6b7b-0be6-4c16-8c8c-e4bd6b173c06 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:23.133620 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a6b7b-0be6-4c16-8c8c-e4bd6b173c06 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:23.140210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54cf13-960c-4ec9-95b0-184728dfc3d7 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-f39a6b7b-0be6-4c16-8c8c-e4bd6b173c06 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:23.147928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:23.148994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:23.149935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:23.149935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:23.149935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:23.149935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:23.149935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:23.149935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:23.149935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:23.149935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:23.149935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:41:23.149935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:23.153717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 411/811] 10.222.0.52 () {56 vars in 1093 bytes} [Thu May 19 13:41:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:23.154557 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a6b7b-0be6-4c16-8c8c-e4bd6b173c06 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:23.178000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f6e84-1964-4c2f-b1ee-e68a1c05225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:23.178809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f6e84-1964-4c2f-b1ee-e68a1c05225c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:23.179116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f6e84-1964-4c2f-b1ee-e68a1c05225c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:23.179391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f6e84-1964-4c2f-b1ee-e68a1c05225c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:23.187075 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f6e84-1964-4c2f-b1ee-e68a1c05225c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:23.188508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f6e84-1964-4c2f-b1ee-e68a1c05225c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-2d5f6e84-1964-4c2f-b1ee-e68a1c05225c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:23.204956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:23.211840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:23.213051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:23.213051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:23.213051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:23.213051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:23.213051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:23.213051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:23.213051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:23.213051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:23.213051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:41:23.213051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:23.213051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 401/812] 10.222.0.52 () {56 vars in 1117 bytes} [Thu May 19 13:41:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:23.213051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fe9d4-6715-4565-9fec-052481874e36 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:23.213051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fe9d4-6715-4565-9fec-052481874e36 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:23.213899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fe9d4-6715-4565-9fec-052481874e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:23.227105 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-5a4fe9d4-6715-4565-9fec-052481874e36 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:23.228319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5a4fe9d4-6715-4565-9fec-052481874e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:23.241425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:23.272161 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:23.277373 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a6b7b-0be6-4c16-8c8c-e4bd6b173c06 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:23.299123 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 402/813] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:23.300022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fe9d4-6715-4565-9fec-052481874e36 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:23.332791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6448695-7a8e-430e-9d4c-935d61bdbe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:23.333572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6448695-7a8e-430e-9d4c-935d61bdbe6e None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:23.333923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6448695-7a8e-430e-9d4c-935d61bdbe6e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:23.334276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6448695-7a8e-430e-9d4c-935d61bdbe6e None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:23.335103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6448695-7a8e-430e-9d4c-935d61bdbe6e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:23.335416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6448695-7a8e-430e-9d4c-935d61bdbe6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:23.336795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6448695-7a8e-430e-9d4c-935d61bdbe6e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:23.342001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 412/814] 10.222.0.52 () {56 vars in 1124 bytes} [Thu May 19 13:41:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:23.342639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6448695-7a8e-430e-9d4c-935d61bdbe6e None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:23.418747 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fe9d4-6715-4565-9fec-052481874e36 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:23.454956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724597a-4ce1-4142-be5d-abaa8313313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:23.455608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724597a-4ce1-4142-be5d-abaa8313313a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:23.456414 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724597a-4ce1-4142-be5d-abaa8313313a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:23.456689 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724597a-4ce1-4142-be5d-abaa8313313a None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:23.457314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6448695-7a8e-430e-9d4c-935d61bdbe6e None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:23.457517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724597a-4ce1-4142-be5d-abaa8313313a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:23.457820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724597a-4ce1-4142-be5d-abaa8313313a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:23.459256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724597a-4ce1-4142-be5d-abaa8313313a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:23.477185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 403/815] 10.222.0.52 () {56 vars in 1133 bytes} [Thu May 19 13:41:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:23.477912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724597a-4ce1-4142-be5d-abaa8313313a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:23.501891 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6978d56-c7b9-40a3-8aa6-839b93a7b43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:23.502679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6978d56-c7b9-40a3-8aa6-839b93a7b43f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:23.503072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6978d56-c7b9-40a3-8aa6-839b93a7b43f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:23.503514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6978d56-c7b9-40a3-8aa6-839b93a7b43f None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:23.504373 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6978d56-c7b9-40a3-8aa6-839b93a7b43f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:23.504755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6978d56-c7b9-40a3-8aa6-839b93a7b43f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:23.506302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6978d56-c7b9-40a3-8aa6-839b93a7b43f None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:23.509982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:23.515341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:23.529854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 413/816] 10.222.0.52 () {56 vars in 1149 bytes} [Thu May 19 13:41:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:23.531062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6978d56-c7b9-40a3-8aa6-839b93a7b43f None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:23.592385 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724597a-4ce1-4142-be5d-abaa8313313a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:23.628549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a4ac8-ce88-426f-868d-9346436dbcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:23.629446 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a4ac8-ce88-426f-868d-9346436dbcdb None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:23.629702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a4ac8-ce88-426f-868d-9346436dbcdb None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:23.629888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a4ac8-ce88-426f-868d-9346436dbcdb None admin] PATH_INFO: `/v3/services/network` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:23.630552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a4ac8-ce88-426f-868d-9346436dbcdb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:23.631930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a4ac8-ce88-426f-868d-9346436dbcdb None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:23.640466 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6978d56-c7b9-40a3-8aa6-839b93a7b43f None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-e01a4ac8-ce88-426f-868d-9346436dbcdb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:23.644504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:23.645472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:41:23.646316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:23.650611 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 404/817] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:23 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:23.651494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a4ac8-ce88-426f-868d-9346436dbcdb None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:23.678293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d70250-5943-4091-a70d-34797f4341a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:23.678992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d70250-5943-4091-a70d-34797f4341a0 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:23.679294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d70250-5943-4091-a70d-34797f4341a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:23.679563 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d70250-5943-4091-a70d-34797f4341a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:23.680312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d70250-5943-4091-a70d-34797f4341a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:23.681683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d70250-5943-4091-a70d-34797f4341a0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:23.704055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 414/818] 10.222.0.52 () {60 vars in 1144 bytes} [Thu May 19 13:41:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:23.704795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d70250-5943-4091-a70d-34797f4341a0 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:23.771871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:23.776092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a4ac8-ce88-426f-868d-9346436dbcdb None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:23.784077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:23.828699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd3ab3-6341-4c8b-95d1-b36751f6f528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:23.829371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd3ab3-6341-4c8b-95d1-b36751f6f528 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:23.829652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd3ab3-6341-4c8b-95d1-b36751f6f528 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:23.829929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd3ab3-6341-4c8b-95d1-b36751f6f528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:23.865514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d70250-5943-4091-a70d-34797f4341a0 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:23.912388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f5bcc-80be-4cfd-892a-708fb12e2c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:23.913191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f5bcc-80be-4cfd-892a-708fb12e2c8a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:23.913477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f5bcc-80be-4cfd-892a-708fb12e2c8a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:23.913762 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f5bcc-80be-4cfd-892a-708fb12e2c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:23.916549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd3ab3-6341-4c8b-95d1-b36751f6f528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:23.918302 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd3ab3-6341-4c8b-95d1-b36751f6f528 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:23.926988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f5bcc-80be-4cfd-892a-708fb12e2c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:23.928451 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f5bcc-80be-4cfd-892a-708fb12e2c8a None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-345f5bcc-80be-4cfd-892a-708fb12e2c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:23.947384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:23.948322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:23.949302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:23.949302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:23.949302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:23.949302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:23.951263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 415/819] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:23.952256 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f5bcc-80be-4cfd-892a-708fb12e2c8a None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:24.024905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:24.040989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 405/820] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:41:24.041859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd3ab3-6341-4c8b-95d1-b36751f6f528 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:24.086195 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f5bcc-80be-4cfd-892a-708fb12e2c8a None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:24.088678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:24.132734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e02b6-1f91-4f1b-9ca3-e7e6360c9524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:24.133647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e02b6-1f91-4f1b-9ca3-e7e6360c9524 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:24.134105 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e02b6-1f91-4f1b-9ca3-e7e6360c9524 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:24.134524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e02b6-1f91-4f1b-9ca3-e7e6360c9524 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:24.135411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e02b6-1f91-4f1b-9ca3-e7e6360c9524 None admin] RBAC: Adding query filter params (name=network) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:24.135859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e02b6-1f91-4f1b-9ca3-e7e6360c9524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:24.137471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e02b6-1f91-4f1b-9ca3-e7e6360c9524 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:24.143151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 416/821] 10.222.0.52 () {56 vars in 1112 bytes} [Thu May 19 13:41:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:24.144391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e02b6-1f91-4f1b-9ca3-e7e6360c9524 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:24.162541 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd3ab3-6341-4c8b-95d1-b36751f6f528 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:24.198759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068e7e3-3ef6-48c1-98f3-8e7e71d77d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:24.199477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068e7e3-3ef6-48c1-98f3-8e7e71d77d0c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:24.199763 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068e7e3-3ef6-48c1-98f3-8e7e71d77d0c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:24.200035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068e7e3-3ef6-48c1-98f3-8e7e71d77d0c None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:24.200855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068e7e3-3ef6-48c1-98f3-8e7e71d77d0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:24.201164 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068e7e3-3ef6-48c1-98f3-8e7e71d77d0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:24.202526 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068e7e3-3ef6-48c1-98f3-8e7e71d77d0c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:24.219268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 406/822] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:24.220157 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068e7e3-3ef6-48c1-98f3-8e7e71d77d0c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:24.256108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e02b6-1f91-4f1b-9ca3-e7e6360c9524 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:24.277981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:24.295652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23be8d-ca7c-46d4-9360-69d26053244b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:24.296766 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23be8d-ca7c-46d4-9360-69d26053244b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:24.297228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23be8d-ca7c-46d4-9360-69d26053244b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:24.297678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23be8d-ca7c-46d4-9360-69d26053244b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:24.309742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23be8d-ca7c-46d4-9360-69d26053244b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:24.311510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23be8d-ca7c-46d4-9360-69d26053244b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:24.326523 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068e7e3-3ef6-48c1-98f3-8e7e71d77d0c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-da23be8d-ca7c-46d4-9360-69d26053244b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:24.330722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:24.332268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:24.333345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:24.333345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:24.333345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:24.333345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:24.335261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 417/823] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:24.336188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23be8d-ca7c-46d4-9360-69d26053244b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:24.357261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:24.368796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e67a1b-5868-46de-aea2-8f8a0ba0d79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:24.369558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e67a1b-5868-46de-aea2-8f8a0ba0d79c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:24.370055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e67a1b-5868-46de-aea2-8f8a0ba0d79c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:24.370410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e67a1b-5868-46de-aea2-8f8a0ba0d79c None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:24.371552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e67a1b-5868-46de-aea2-8f8a0ba0d79c None admin] RBAC: Adding query filter params (type=network) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:24.372187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e67a1b-5868-46de-aea2-8f8a0ba0d79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:24.374060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e67a1b-5868-46de-aea2-8f8a0ba0d79c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:24.381550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 407/824] 10.222.0.52 () {56 vars in 1112 bytes} [Thu May 19 13:41:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:24.382556 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e67a1b-5868-46de-aea2-8f8a0ba0d79c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:24.472476 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23be8d-ca7c-46d4-9360-69d26053244b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:24.516176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26d1cb-f82e-45e1-afd2-363eb511ad30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:24.516972 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26d1cb-f82e-45e1-afd2-363eb511ad30 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:24.517253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26d1cb-f82e-45e1-afd2-363eb511ad30 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:24.517527 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26d1cb-f82e-45e1-afd2-363eb511ad30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:24.525431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e67a1b-5868-46de-aea2-8f8a0ba0d79c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:24.527087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26d1cb-f82e-45e1-afd2-363eb511ad30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:24.529813 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26d1cb-f82e-45e1-afd2-363eb511ad30 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-cf26d1cb-f82e-45e1-afd2-363eb511ad30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:24.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:24.550351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:24.551281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:24.551281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:24.551281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:24.551281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:24.551281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:24.551281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:24.551281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:24.551281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:24.551695 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:24.558509 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 418/825] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:24.559692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26d1cb-f82e-45e1-afd2-363eb511ad30 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:24.572656 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd33fdb-d50f-4d40-aca6-d1318fa1decc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:24.573374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd33fdb-d50f-4d40-aca6-d1318fa1decc None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:24.573724 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd33fdb-d50f-4d40-aca6-d1318fa1decc None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:24.574074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd33fdb-d50f-4d40-aca6-d1318fa1decc None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:24.574909 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd33fdb-d50f-4d40-aca6-d1318fa1decc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:24.575305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd33fdb-d50f-4d40-aca6-d1318fa1decc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:24.576904 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd33fdb-d50f-4d40-aca6-d1318fa1decc None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:24.595012 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 408/826] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:24.595012 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd33fdb-d50f-4d40-aca6-d1318fa1decc None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:24.650082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:24.673651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26d1cb-f82e-45e1-afd2-363eb511ad30 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:24.704494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd33fdb-d50f-4d40-aca6-d1318fa1decc None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:24.710822 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bad58d-687e-4796-ba8e-cf1afae13f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:24.711841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bad58d-687e-4796-ba8e-cf1afae13f3e None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:24.712308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bad58d-687e-4796-ba8e-cf1afae13f3e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:24.712837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bad58d-687e-4796-ba8e-cf1afae13f3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:24.713790 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bad58d-687e-4796-ba8e-cf1afae13f3e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:24.715391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bad58d-687e-4796-ba8e-cf1afae13f3e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:24.738687 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 419/827] 10.222.0.52 () {60 vars in 1144 bytes} [Thu May 19 13:41:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:24.740350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bad58d-687e-4796-ba8e-cf1afae13f3e None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:24.750036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196178b2-e78d-4d41-8131-cddc34970c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:24.750772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196178b2-e78d-4d41-8131-cddc34970c50 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:24.751692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196178b2-e78d-4d41-8131-cddc34970c50 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:24.751692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196178b2-e78d-4d41-8131-cddc34970c50 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:24.752328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196178b2-e78d-4d41-8131-cddc34970c50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:24.752700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196178b2-e78d-4d41-8131-cddc34970c50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:24.755000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196178b2-e78d-4d41-8131-cddc34970c50 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:24.769359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 409/828] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:24.776050 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196178b2-e78d-4d41-8131-cddc34970c50 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:24.862725 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:24.948192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bad58d-687e-4796-ba8e-cf1afae13f3e None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:24.959872 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196178b2-e78d-4d41-8131-cddc34970c50 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:24.988001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355483f7-a858-4685-97bb-1f4f13dc40f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:24.988800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355483f7-a858-4685-97bb-1f4f13dc40f1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:24.989080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355483f7-a858-4685-97bb-1f4f13dc40f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:24.989346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355483f7-a858-4685-97bb-1f4f13dc40f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:24.991644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:24.997672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355483f7-a858-4685-97bb-1f4f13dc40f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:24.999142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355483f7-a858-4685-97bb-1f4f13dc40f1 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:25.005171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4dbc3c-b517-43eb-a792-775ef3f0aca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:25.006278 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dbc3c-b517-43eb-a792-775ef3f0aca8 None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:25.006715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dbc3c-b517-43eb-a792-775ef3f0aca8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:25.007185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dbc3c-b517-43eb-a792-775ef3f0aca8 None admin] PATH_INFO: `/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/a989e5551c1640ebbde8768247441627/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-355483f7-a858-4685-97bb-1f4f13dc40f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:25.016469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:25.017568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:25.019507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:25.019507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:25.019507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:25.019507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:25.019507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:25.019507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:25.019507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:25.019507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:25.019507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 420/829] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:25.024725 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355483f7-a858-4685-97bb-1f4f13dc40f1 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:25.029260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4dbc3c-b517-43eb-a792-775ef3f0aca8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fc6fd42a31942688f570f1ce86be510, user_id=a989e5551c1640ebbde8768247441627, role_id=c0cbd2d231594ea2bf7bb013fbf9ad1f)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:25.031134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4dbc3c-b517-43eb-a792-775ef3f0aca8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:25.052759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 410/830] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:41:24 2022] PUT /identity/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/a989e5551c1640ebbde8768247441627/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:41:25.136647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6ea04-7971-417d-9128-4a7d3d9bee32 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:25.137017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6ea04-7971-417d-9128-4a7d3d9bee32 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:25.137219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6ea04-7971-417d-9128-4a7d3d9bee32 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:25.138820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 411/831] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:41:25.149168 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dec77f-263f-4bc3-93e9-52aafc40ddf1 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:25.149517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dec77f-263f-4bc3-93e9-52aafc40ddf1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:25.149729 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dec77f-263f-4bc3-93e9-52aafc40ddf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:25.166545 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355483f7-a858-4685-97bb-1f4f13dc40f1 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:25.180257 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-95dec77f-263f-4bc3-93e9-52aafc40ddf1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:25.182248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-95dec77f-263f-4bc3-93e9-52aafc40ddf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:25.219283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cff3f5-e679-46bf-b0ad-3bbfdceee5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:25.219970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cff3f5-e679-46bf-b0ad-3bbfdceee5cb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:25.223472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cff3f5-e679-46bf-b0ad-3bbfdceee5cb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:25.223803 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cff3f5-e679-46bf-b0ad-3bbfdceee5cb None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:25.224659 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cff3f5-e679-46bf-b0ad-3bbfdceee5cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:25.224979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cff3f5-e679-46bf-b0ad-3bbfdceee5cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:25.226513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cff3f5-e679-46bf-b0ad-3bbfdceee5cb None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:25.241215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 421/832] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:25.247076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cff3f5-e679-46bf-b0ad-3bbfdceee5cb None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:25.289025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 412/833] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:25.312466 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dec77f-263f-4bc3-93e9-52aafc40ddf1 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:25.322653 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:25.333876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:25.374352 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cff3f5-e679-46bf-b0ad-3bbfdceee5cb None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:25.426224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32423e88-9e19-4e98-8efb-44eee462817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:25.427259 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32423e88-9e19-4e98-8efb-44eee462817c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:25.427799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32423e88-9e19-4e98-8efb-44eee462817c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:25.428169 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32423e88-9e19-4e98-8efb-44eee462817c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:25.429112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32423e88-9e19-4e98-8efb-44eee462817c None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=4f6093d0c1fc41d98c58bace81e3d8e5) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:25.429531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32423e88-9e19-4e98-8efb-44eee462817c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:25.431175 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32423e88-9e19-4e98-8efb-44eee462817c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:25.437752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 422/834] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:25 2022] GET /identity/v3/role_assignments?user.id=4f6093d0c1fc41d98c58bace81e3d8e5&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=0893bb944dd94634accd3906aa391cd2 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:25.439661 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dec77f-263f-4bc3-93e9-52aafc40ddf1 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:25.481261 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001e692-e559-490d-8985-7ebc5aeb1a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:25.482444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001e692-e559-490d-8985-7ebc5aeb1a44 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:25.482980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001e692-e559-490d-8985-7ebc5aeb1a44 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:25.483474 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001e692-e559-490d-8985-7ebc5aeb1a44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:25.497244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001e692-e559-490d-8985-7ebc5aeb1a44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-2001e692-e559-490d-8985-7ebc5aeb1a44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:25.506959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:25.507985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:25.508843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:25.508843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:25.508843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:25.508843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:25.508843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:25.508843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:25.508843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:25.508843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:25.508843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:25.508843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:25.508843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:25.508843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:25.511890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 413/835] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:41:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:25.520577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32423e88-9e19-4e98-8efb-44eee462817c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:25.641024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32423e88-9e19-4e98-8efb-44eee462817c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:25.658702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:25.682631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e02c7a-78ce-4e04-8b1f-55e0f0ee6e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:25.683325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e02c7a-78ce-4e04-8b1f-55e0f0ee6e12 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:25.683608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e02c7a-78ce-4e04-8b1f-55e0f0ee6e12 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:25.683883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e02c7a-78ce-4e04-8b1f-55e0f0ee6e12 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:25.684630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e02c7a-78ce-4e04-8b1f-55e0f0ee6e12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:25.684941 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e02c7a-78ce-4e04-8b1f-55e0f0ee6e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:25.686297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e02c7a-78ce-4e04-8b1f-55e0f0ee6e12 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:25.696460 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 423/836] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:41:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:25.702326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001e692-e559-490d-8985-7ebc5aeb1a44 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:25.784864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:25.817410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001e692-e559-490d-8985-7ebc5aeb1a44 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:25.851179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4753ab-9c8b-49ea-96a6-7f5a145189cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:25.851976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4753ab-9c8b-49ea-96a6-7f5a145189cc None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:25.852259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4753ab-9c8b-49ea-96a6-7f5a145189cc None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:25.852532 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4753ab-9c8b-49ea-96a6-7f5a145189cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:25.864203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4753ab-9c8b-49ea-96a6-7f5a145189cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:25.865648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4753ab-9c8b-49ea-96a6-7f5a145189cc None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-0f4753ab-9c8b-49ea-96a6-7f5a145189cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:25.883966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:25.884912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:25.885802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:25.885802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:25.885802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:25.885802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:25.889791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 414/837] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:25.891174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e02c7a-78ce-4e04-8b1f-55e0f0ee6e12 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:26.001923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e02c7a-78ce-4e04-8b1f-55e0f0ee6e12 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:26.041900 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab165d2-f74e-4588-9f86-345bb782f99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:26.042855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab165d2-f74e-4588-9f86-345bb782f99e None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:26.043322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab165d2-f74e-4588-9f86-345bb782f99e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:26.043752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab165d2-f74e-4588-9f86-345bb782f99e None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:26.044670 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab165d2-f74e-4588-9f86-345bb782f99e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:26.045179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab165d2-f74e-4588-9f86-345bb782f99e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:26.046799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab165d2-f74e-4588-9f86-345bb782f99e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:26.064159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 424/838] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:26.070918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4753ab-9c8b-49ea-96a6-7f5a145189cc None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:26.177669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4753ab-9c8b-49ea-96a6-7f5a145189cc None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:26.207212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:26.213368 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c0f25-3585-45b9-87f9-34c9606ef949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:26.214165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c0f25-3585-45b9-87f9-34c9606ef949 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:26.214451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c0f25-3585-45b9-87f9-34c9606ef949 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:26.214726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c0f25-3585-45b9-87f9-34c9606ef949 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:26.226356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d3540-783b-47e1-a36d-60dbca813e95 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:26.226849 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d3540-783b-47e1-a36d-60dbca813e95 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:26.226956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d3540-783b-47e1-a36d-60dbca813e95 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:26.228396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 425/839] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:26.231597 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c0f25-3585-45b9-87f9-34c9606ef949 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:26.233168 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c0f25-3585-45b9-87f9-34c9606ef949 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:26.234001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc33db1-164c-4bef-a763-bf23ec2b3612 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:26.234491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc33db1-164c-4bef-a763-bf23ec2b3612 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:26.235008 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc33db1-164c-4bef-a763-bf23ec2b3612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-ba1c0f25-3585-45b9-87f9-34c9606ef949 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:26.249481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:26.250478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:26.251469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:26.251469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:26.251469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:26.251469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:26.251469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:26.251469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:26.251469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:26.251469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:26.251469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 13:41:26.251469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:26.255005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 415/840] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:41:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:26.257050 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-bfc33db1-164c-4bef-a763-bf23ec2b3612 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:26.257396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c0f25-3585-45b9-87f9-34c9606ef949 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:26.258686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-bfc33db1-164c-4bef-a763-bf23ec2b3612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:26.296362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:26.343263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 426/841] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:26.367492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc33db1-164c-4bef-a763-bf23ec2b3612 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:26.372790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c0f25-3585-45b9-87f9-34c9606ef949 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:26.417606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75eb2bb-1a4a-47d3-9cec-2903dafc9b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:26.418302 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eb2bb-1a4a-47d3-9cec-2903dafc9b72 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:26.418587 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eb2bb-1a4a-47d3-9cec-2903dafc9b72 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:26.418901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eb2bb-1a4a-47d3-9cec-2903dafc9b72 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:26.419892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75eb2bb-1a4a-47d3-9cec-2903dafc9b72 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:26.420204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75eb2bb-1a4a-47d3-9cec-2903dafc9b72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:26.421711 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75eb2bb-1a4a-47d3-9cec-2903dafc9b72 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:26.447880 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 416/842] 10.222.0.52 () {56 vars in 1135 bytes} [Thu May 19 13:41:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:26.449190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75eb2bb-1a4a-47d3-9cec-2903dafc9b72 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:26.505771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:26.508971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc33db1-164c-4bef-a763-bf23ec2b3612 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:26.555851 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0058c05-8e78-4c68-85e2-c3f68d72deeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:26.556658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058c05-8e78-4c68-85e2-c3f68d72deeb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:26.556941 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058c05-8e78-4c68-85e2-c3f68d72deeb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:26.557220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058c05-8e78-4c68-85e2-c3f68d72deeb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:26.558159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0058c05-8e78-4c68-85e2-c3f68d72deeb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:26.559825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0058c05-8e78-4c68-85e2-c3f68d72deeb None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:26.570438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75eb2bb-1a4a-47d3-9cec-2903dafc9b72 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-d0058c05-8e78-4c68-85e2-c3f68d72deeb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:26.576074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:26.577110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:41:26.577940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:26.583063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 427/843] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:41:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:26.584483 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0058c05-8e78-4c68-85e2-c3f68d72deeb None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:26.618038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce403f4-6993-4413-b56c-7ef9edb7494e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:26.619303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce403f4-6993-4413-b56c-7ef9edb7494e None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:26.620048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce403f4-6993-4413-b56c-7ef9edb7494e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:26.620849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce403f4-6993-4413-b56c-7ef9edb7494e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:26.628099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:26.699888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce403f4-6993-4413-b56c-7ef9edb7494e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:26.701390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce403f4-6993-4413-b56c-7ef9edb7494e None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:26.724379 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0058c05-8e78-4c68-85e2-c3f68d72deeb None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:26.753399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:26.768086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f687580-a402-45d3-b151-6389e9b036ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:26.769341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f687580-a402-45d3-b151-6389e9b036ea None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:26.769730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f687580-a402-45d3-b151-6389e9b036ea None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:26.770376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f687580-a402-45d3-b151-6389e9b036ea None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:26.771231 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f687580-a402-45d3-b151-6389e9b036ea None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:26.771638 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f687580-a402-45d3-b151-6389e9b036ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:26.773879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f687580-a402-45d3-b151-6389e9b036ea None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:26.783015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 428/844] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:41:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:26.787060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f687580-a402-45d3-b151-6389e9b036ea None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:26.823660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 417/845] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:26 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:41:26.829120 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce403f4-6993-4413-b56c-7ef9edb7494e None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:26.889097 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:26.894206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f687580-a402-45d3-b151-6389e9b036ea None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:26.929671 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126edae8-ecd0-4aa9-86ea-2f731442a3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:26.930614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126edae8-ecd0-4aa9-86ea-2f731442a3e5 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:26.931160 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126edae8-ecd0-4aa9-86ea-2f731442a3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:26.931579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126edae8-ecd0-4aa9-86ea-2f731442a3e5 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:26.932425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126edae8-ecd0-4aa9-86ea-2f731442a3e5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:26.932874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126edae8-ecd0-4aa9-86ea-2f731442a3e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:26.934377 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126edae8-ecd0-4aa9-86ea-2f731442a3e5 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:26.939653 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 429/846] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:41:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:26.945311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce403f4-6993-4413-b56c-7ef9edb7494e None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:26.987501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64660a08-732a-4fd1-ac6b-f67933069a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:26.988869 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64660a08-732a-4fd1-ac6b-f67933069a47 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:26.989382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64660a08-732a-4fd1-ac6b-f67933069a47 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:26.989872 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64660a08-732a-4fd1-ac6b-f67933069a47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:27.003408 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64660a08-732a-4fd1-ac6b-f67933069a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:27.005318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64660a08-732a-4fd1-ac6b-f67933069a47 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:27.008866 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-64660a08-732a-4fd1-ac6b-f67933069a47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:27.027306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:27.028218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:27.029098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:27.029098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:27.029098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:27.029098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:27.031476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 418/847] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:27.040171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126edae8-ecd0-4aa9-86ea-2f731442a3e5 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:27.152678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126edae8-ecd0-4aa9-86ea-2f731442a3e5 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:27.189025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261b9a5-80b3-4fe6-afbe-4a6abaf1bcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:27.189997 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261b9a5-80b3-4fe6-afbe-4a6abaf1bcc0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:27.191126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261b9a5-80b3-4fe6-afbe-4a6abaf1bcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:27.191587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261b9a5-80b3-4fe6-afbe-4a6abaf1bcc0 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:27.192490 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261b9a5-80b3-4fe6-afbe-4a6abaf1bcc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:27.193057 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261b9a5-80b3-4fe6-afbe-4a6abaf1bcc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:27.194725 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261b9a5-80b3-4fe6-afbe-4a6abaf1bcc0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:27.210980 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 430/848] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:27.213552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64660a08-732a-4fd1-ac6b-f67933069a47 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:27.324731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64660a08-732a-4fd1-ac6b-f67933069a47 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:27.360901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bb868-906d-4705-9963-7891032faee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:27.361698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bb868-906d-4705-9963-7891032faee2 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:27.362570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bb868-906d-4705-9963-7891032faee2 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:27.362881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bb868-906d-4705-9963-7891032faee2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:27.369924 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bb868-906d-4705-9963-7891032faee2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:27.371381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bb868-906d-4705-9963-7891032faee2 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-b97bb868-906d-4705-9963-7891032faee2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:27.387349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:27.388539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:27.389633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:27.389633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:27.389633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:27.389633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:27.389633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:27.389633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:27.389633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:27.389633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:27.389633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 419/849] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:27.402343 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261b9a5-80b3-4fe6-afbe-4a6abaf1bcc0 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:27.430119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:27.524486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261b9a5-80b3-4fe6-afbe-4a6abaf1bcc0 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:27.564542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec3f3b-003e-48e5-9df7-565f577d6f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:27.566088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec3f3b-003e-48e5-9df7-565f577d6f57 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:27.566581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec3f3b-003e-48e5-9df7-565f577d6f57 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:27.567145 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec3f3b-003e-48e5-9df7-565f577d6f57 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:27.568140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec3f3b-003e-48e5-9df7-565f577d6f57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:27.568706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec3f3b-003e-48e5-9df7-565f577d6f57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:27.570339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec3f3b-003e-48e5-9df7-565f577d6f57 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:27.588776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 431/850] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:27.590391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bb868-906d-4705-9963-7891032faee2 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:27.639472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093c569-0ce4-40ef-a999-bd3b555eaa9f None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:27.639805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093c569-0ce4-40ef-a999-bd3b555eaa9f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:27.640032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093c569-0ce4-40ef-a999-bd3b555eaa9f None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:27.642273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 432/851] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:41:27.648218 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d25b1-359a-49a4-bd46-689de1636327 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:27.648505 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d25b1-359a-49a4-bd46-689de1636327 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:27.648788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d25b1-359a-49a4-bd46-689de1636327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:27.653937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:27.668506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-671d25b1-359a-49a4-bd46-689de1636327 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:27.669685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-671d25b1-359a-49a4-bd46-689de1636327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:27.711988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bb868-906d-4705-9963-7891032faee2 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:27.754456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2085e25-252a-470f-b180-23f5aa3aa3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:27.755168 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2085e25-252a-470f-b180-23f5aa3aa3f8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:27.755421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2085e25-252a-470f-b180-23f5aa3aa3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:27.755693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2085e25-252a-470f-b180-23f5aa3aa3f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:27.756631 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2085e25-252a-470f-b180-23f5aa3aa3f8 None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=8554ff40bbcb4fbb801d854d6527a6d2) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:27.756891 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 433/852] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:27.763414 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2085e25-252a-470f-b180-23f5aa3aa3f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:27.767223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2085e25-252a-470f-b180-23f5aa3aa3f8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:27.774660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 420/853] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:27 2022] GET /identity/v3/role_assignments?user.id=8554ff40bbcb4fbb801d854d6527a6d2&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=0893bb944dd94634accd3906aa391cd2 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:27.783393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d25b1-359a-49a4-bd46-689de1636327 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:27.805731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:27.911059 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d25b1-359a-49a4-bd46-689de1636327 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:27.938589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb489d5-9d15-424b-9427-bf10794887f9 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:27.938589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb489d5-9d15-424b-9427-bf10794887f9 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:27.938853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb489d5-9d15-424b-9427-bf10794887f9 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:27.940956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 421/854] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:27.946704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae435cf-bba8-4221-bd62-680cbbbd9cb3 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:27.947027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae435cf-bba8-4221-bd62-680cbbbd9cb3 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:27.947292 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae435cf-bba8-4221-bd62-680cbbbd9cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:27.952177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27babd-1899-4791-9578-791ea9ec3f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:27.953423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27babd-1899-4791-9578-791ea9ec3f2f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:27.954039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27babd-1899-4791-9578-791ea9ec3f2f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:27.954315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27babd-1899-4791-9578-791ea9ec3f2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:27.967100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27babd-1899-4791-9578-791ea9ec3f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:27.967933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-cae435cf-bba8-4221-bd62-680cbbbd9cb3 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:27.969374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-cae435cf-bba8-4221-bd62-680cbbbd9cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-6f27babd-1899-4791-9578-791ea9ec3f2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:27.975825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:27.976765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:27.977605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:27.977605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:27.977605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:27.977605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:27.977605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:27.977605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:27.977605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:27.977605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:27.977605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:27.977605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:27.977605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:41:27.977605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:27.977605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 434/855] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:41:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:27.983779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27babd-1899-4791-9578-791ea9ec3f2f None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:28.054911 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 422/856] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:28.076424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:28.077741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae435cf-bba8-4221-bd62-680cbbbd9cb3 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:28.111394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27babd-1899-4791-9578-791ea9ec3f2f None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:28.152528 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:28.152730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc513df4-2c4c-4a89-9bbe-79393c218dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:28.153269 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc513df4-2c4c-4a89-9bbe-79393c218dd9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:28.153935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc513df4-2c4c-4a89-9bbe-79393c218dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:28.154218 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc513df4-2c4c-4a89-9bbe-79393c218dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:28.155017 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc513df4-2c4c-4a89-9bbe-79393c218dd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:28.155488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc513df4-2c4c-4a89-9bbe-79393c218dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:28.156954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc513df4-2c4c-4a89-9bbe-79393c218dd9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:28.166998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 435/857] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:41:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:28.172302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc513df4-2c4c-4a89-9bbe-79393c218dd9 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:28.220927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae435cf-bba8-4221-bd62-680cbbbd9cb3 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:28.258825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195560ac-ebb9-495a-ae2b-ef20db9411b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:28.259675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195560ac-ebb9-495a-ae2b-ef20db9411b3 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:28.260051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195560ac-ebb9-495a-ae2b-ef20db9411b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:28.260323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195560ac-ebb9-495a-ae2b-ef20db9411b3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:28.260971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195560ac-ebb9-495a-ae2b-ef20db9411b3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:28.262435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195560ac-ebb9-495a-ae2b-ef20db9411b3 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-195560ac-ebb9-495a-ae2b-ef20db9411b3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:28.276601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:28.277634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:28.278746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 423/858] 10.222.0.52 () {56 vars in 1132 bytes} [Thu May 19 13:41:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:28.283855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195560ac-ebb9-495a-ae2b-ef20db9411b3 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:28.284589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc513df4-2c4c-4a89-9bbe-79393c218dd9 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:28.323643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b98c19-907a-4fbc-80e1-6b057c1053d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:28.325284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b98c19-907a-4fbc-80e1-6b057c1053d4 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:28.325623 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b98c19-907a-4fbc-80e1-6b057c1053d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:28.325912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b98c19-907a-4fbc-80e1-6b057c1053d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:28.327962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:28.342633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b98c19-907a-4fbc-80e1-6b057c1053d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:28.344178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b98c19-907a-4fbc-80e1-6b057c1053d4 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-48b98c19-907a-4fbc-80e1-6b057c1053d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:28.366923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:28.367888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:28.368935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:28.368935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:28.368935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:28.368935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:28.368935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 436/859] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:28.373905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b98c19-907a-4fbc-80e1-6b057c1053d4 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:28.415654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195560ac-ebb9-495a-ae2b-ef20db9411b3 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:28.430426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:28.457746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f19e2-aed4-4914-bde8-12ea99631334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:28.458709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f19e2-aed4-4914-bde8-12ea99631334 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:28.459340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f19e2-aed4-4914-bde8-12ea99631334 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:28.459791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f19e2-aed4-4914-bde8-12ea99631334 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:28.461172 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f19e2-aed4-4914-bde8-12ea99631334 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:28.461687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f19e2-aed4-4914-bde8-12ea99631334 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:28.463666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f19e2-aed4-4914-bde8-12ea99631334 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:28.469688 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 424/860] 10.222.0.52 () {56 vars in 1126 bytes} [Thu May 19 13:41:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:28.476761 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f19e2-aed4-4914-bde8-12ea99631334 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:28.504399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b98c19-907a-4fbc-80e1-6b057c1053d4 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:28.546445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d778ef-85ed-4ff5-a0b2-0c0bda31f230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:28.548034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d778ef-85ed-4ff5-a0b2-0c0bda31f230 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:28.548508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d778ef-85ed-4ff5-a0b2-0c0bda31f230 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:28.549193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d778ef-85ed-4ff5-a0b2-0c0bda31f230 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:28.550944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d778ef-85ed-4ff5-a0b2-0c0bda31f230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:28.551749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d778ef-85ed-4ff5-a0b2-0c0bda31f230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:28.553316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d778ef-85ed-4ff5-a0b2-0c0bda31f230 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:28.569556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 437/861] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:28.574491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d778ef-85ed-4ff5-a0b2-0c0bda31f230 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:28.605789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f19e2-aed4-4914-bde8-12ea99631334 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:28.631340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:28.652032 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefe27a-2f1f-45fc-bd29-b37cb9169ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:28.653116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefe27a-2f1f-45fc-bd29-b37cb9169ed6 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:28.653610 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefe27a-2f1f-45fc-bd29-b37cb9169ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:28.654819 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefe27a-2f1f-45fc-bd29-b37cb9169ed6 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:28.655741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefe27a-2f1f-45fc-bd29-b37cb9169ed6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:28.656198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefe27a-2f1f-45fc-bd29-b37cb9169ed6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:28.657711 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefe27a-2f1f-45fc-bd29-b37cb9169ed6 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:28.664483 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 425/862] 10.222.0.52 () {56 vars in 1126 bytes} [Thu May 19 13:41:28 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:28.695053 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d778ef-85ed-4ff5-a0b2-0c0bda31f230 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:28.733967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4f2cf-0ac0-4ea7-8489-007f50b5aa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:28.735313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4f2cf-0ac0-4ea7-8489-007f50b5aa3b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:28.735814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4f2cf-0ac0-4ea7-8489-007f50b5aa3b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:28.736299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4f2cf-0ac0-4ea7-8489-007f50b5aa3b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:28.746509 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4f2cf-0ac0-4ea7-8489-007f50b5aa3b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:28.748661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4f2cf-0ac0-4ea7-8489-007f50b5aa3b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-e7e4f2cf-0ac0-4ea7-8489-007f50b5aa3b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:28.765142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:28.766141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:28.767100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:28.767100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:28.767100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:28.767100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:28.767100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:28.767100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:28.767100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:28.767100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:28.767100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:41:28.767100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:28.770625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 438/863] 10.222.0.52 () {56 vars in 1117 bytes} [Thu May 19 13:41:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:28.782900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefe27a-2f1f-45fc-bd29-b37cb9169ed6 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:28.831968 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:28.894145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefe27a-2f1f-45fc-bd29-b37cb9169ed6 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:28.929708 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68aabd-99ee-4c40-8ea2-615f5e0d6a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:28.930678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68aabd-99ee-4c40-8ea2-615f5e0d6a3c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:28.931139 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68aabd-99ee-4c40-8ea2-615f5e0d6a3c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:28.931561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68aabd-99ee-4c40-8ea2-615f5e0d6a3c None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:28.932565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68aabd-99ee-4c40-8ea2-615f5e0d6a3c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:28.933023 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68aabd-99ee-4c40-8ea2-615f5e0d6a3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:28.934586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68aabd-99ee-4c40-8ea2-615f5e0d6a3c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:28.953612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 426/864] 10.222.0.52 () {56 vars in 1149 bytes} [Thu May 19 13:41:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:28.960198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4f2cf-0ac0-4ea7-8489-007f50b5aa3b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:29.062885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4f2cf-0ac0-4ea7-8489-007f50b5aa3b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:29.072016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:29.100247 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c2195-a239-4b48-ace6-15e52f2bc5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:29.100892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c2195-a239-4b48-ace6-15e52f2bc5a0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:29.101174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c2195-a239-4b48-ace6-15e52f2bc5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:29.101448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c2195-a239-4b48-ace6-15e52f2bc5a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:29.177097 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c2195-a239-4b48-ace6-15e52f2bc5a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:29.178558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c2195-a239-4b48-ace6-15e52f2bc5a0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:29.281643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 439/865] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:41:29.287969 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68aabd-99ee-4c40-8ea2-615f5e0d6a3c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:29.324771 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947e91f-a0e7-4b42-a37b-dd89a9e7e0eb None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:29.325045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947e91f-a0e7-4b42-a37b-dd89a9e7e0eb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:29.325337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947e91f-a0e7-4b42-a37b-dd89a9e7e0eb None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:29.326950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 440/866] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:29.336273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2293e73-932b-425e-a6d7-c9fa8b8a970a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:29.336692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2293e73-932b-425e-a6d7-c9fa8b8a970a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:29.337036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2293e73-932b-425e-a6d7-c9fa8b8a970a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:29.349065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:29.355539 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-d2293e73-932b-425e-a6d7-c9fa8b8a970a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:29.356836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d2293e73-932b-425e-a6d7-c9fa8b8a970a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:29.424760 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:29.429739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68aabd-99ee-4c40-8ea2-615f5e0d6a3c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:29.436019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 441/867] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:29.458387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2293e73-932b-425e-a6d7-c9fa8b8a970a None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:29.468383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd754b-6f26-4efc-a008-0442482591fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:29.469465 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd754b-6f26-4efc-a008-0442482591fb None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:29.469965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd754b-6f26-4efc-a008-0442482591fb None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:29.470401 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd754b-6f26-4efc-a008-0442482591fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:29.482380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd754b-6f26-4efc-a008-0442482591fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:29.484062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd754b-6f26-4efc-a008-0442482591fb None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-40dd754b-6f26-4efc-a008-0442482591fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:29.505516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:29.506670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:29.507713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:29.507713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:29.507713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:29.507713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:29.509740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 427/868] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:29.515193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd754b-6f26-4efc-a008-0442482591fb None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:29.567551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2293e73-932b-425e-a6d7-c9fa8b8a970a None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:29.604842 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e1b04-e7be-4a7c-acb9-1efbb0acb623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:29.605494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e1b04-e7be-4a7c-acb9-1efbb0acb623 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:29.605778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e1b04-e7be-4a7c-acb9-1efbb0acb623 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:29.606054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e1b04-e7be-4a7c-acb9-1efbb0acb623 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:29.606883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e1b04-e7be-4a7c-acb9-1efbb0acb623 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:29.608294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e1b04-e7be-4a7c-acb9-1efbb0acb623 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:29.613136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:29.630991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 442/869] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:41:29 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:41:29.653674 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd754b-6f26-4efc-a008-0442482591fb None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:29.722898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b238b9-a0c0-41ba-b772-2a09d3225079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:29.722898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b238b9-a0c0-41ba-b772-2a09d3225079 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:29.722898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b238b9-a0c0-41ba-b772-2a09d3225079 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:29.722898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b238b9-a0c0-41ba-b772-2a09d3225079 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:29.723578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b238b9-a0c0-41ba-b772-2a09d3225079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:29.723578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b238b9-a0c0-41ba-b772-2a09d3225079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:29.723578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b238b9-a0c0-41ba-b772-2a09d3225079 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:29.739630 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 428/870] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:29.745533 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e1b04-e7be-4a7c-acb9-1efbb0acb623 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:29.870444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e1b04-e7be-4a7c-acb9-1efbb0acb623 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:29.910127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5fc6d-3479-4c66-96eb-d79d5ffa5b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:29.911247 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5fc6d-3479-4c66-96eb-d79d5ffa5b45 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:29.911704 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5fc6d-3479-4c66-96eb-d79d5ffa5b45 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:29.912135 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5fc6d-3479-4c66-96eb-d79d5ffa5b45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:29.919598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5fc6d-3479-4c66-96eb-d79d5ffa5b45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:29.921203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5fc6d-3479-4c66-96eb-d79d5ffa5b45 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-41e5fc6d-3479-4c66-96eb-d79d5ffa5b45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:29.936528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:29.937713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:29.938916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:29.938916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:29.938916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:29.938916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:29.938916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:29.938916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:29.938916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:29.938916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:29.941136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 443/871] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:29.947187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b238b9-a0c0-41ba-b772-2a09d3225079 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:29.964036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:30.079262 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b238b9-a0c0-41ba-b772-2a09d3225079 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:30.128031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a226f-d4cc-4d77-9c9b-ae577487ecdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:30.128565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a226f-d4cc-4d77-9c9b-ae577487ecdb None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:30.128850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a226f-d4cc-4d77-9c9b-ae577487ecdb None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:30.129130 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a226f-d4cc-4d77-9c9b-ae577487ecdb None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:30.129914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a226f-d4cc-4d77-9c9b-ae577487ecdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:30.130223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a226f-d4cc-4d77-9c9b-ae577487ecdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:30.131661 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a226f-d4cc-4d77-9c9b-ae577487ecdb None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:30.144797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 429/872] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:30.150315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5fc6d-3479-4c66-96eb-d79d5ffa5b45 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:30.209301 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:30.255448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5fc6d-3479-4c66-96eb-d79d5ffa5b45 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:30.256182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad40fc4-61ed-49b8-b355-ac9f766ac8ca None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:30.256703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad40fc4-61ed-49b8-b355-ac9f766ac8ca None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:30.257186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad40fc4-61ed-49b8-b355-ac9f766ac8ca None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:30.260079 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 430/873] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:30.264267 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eb88b-66d4-4c60-bd85-12b97dc55d16 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:30.264797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eb88b-66d4-4c60-bd85-12b97dc55d16 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:30.265302 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eb88b-66d4-4c60-bd85-12b97dc55d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:30.286639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-722eb88b-66d4-4c60-bd85-12b97dc55d16 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:30.288298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-722eb88b-66d4-4c60-bd85-12b97dc55d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:30.290990 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad277b-01d0-4a63-b610-e81ade041d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:30.291645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad277b-01d0-4a63-b610-e81ade041d23 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:30.291926 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad277b-01d0-4a63-b610-e81ade041d23 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:30.292194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad277b-01d0-4a63-b610-e81ade041d23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:30.293025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad277b-01d0-4a63-b610-e81ade041d23 None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=a989e5551c1640ebbde8768247441627) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:30.293335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad277b-01d0-4a63-b610-e81ade041d23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:30.294750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad277b-01d0-4a63-b610-e81ade041d23 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:30.301698 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 444/874] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:30 2022] GET /identity/v3/role_assignments?user.id=a989e5551c1640ebbde8768247441627&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:30.368211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:30.380242 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 431/875] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:30 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:30.407789 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad277b-01d0-4a63-b610-e81ade041d23 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:30.527277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad277b-01d0-4a63-b610-e81ade041d23 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:30.551493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:30.570275 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc66d6-b33c-4230-b114-c4978795e578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:30.571128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc66d6-b33c-4230-b114-c4978795e578 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:30.571412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc66d6-b33c-4230-b114-c4978795e578 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:30.571685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc66d6-b33c-4230-b114-c4978795e578 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:30.572342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc66d6-b33c-4230-b114-c4978795e578 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:30.573719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc66d6-b33c-4230-b114-c4978795e578 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-9edc66d6-b33c-4230-b114-c4978795e578 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:30.586548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:30.587630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:30.588616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 445/876] 10.222.0.52 () {56 vars in 1126 bytes} [Thu May 19 13:41:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:30.596828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722eb88b-66d4-4c60-bd85-12b97dc55d16 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:30.710490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722eb88b-66d4-4c60-bd85-12b97dc55d16 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:30.747040 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c60b02-1631-4c8e-9921-60ed555de267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:30.748114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c60b02-1631-4c8e-9921-60ed555de267 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:30.748596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c60b02-1631-4c8e-9921-60ed555de267 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:30.749073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c60b02-1631-4c8e-9921-60ed555de267 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:30.750099 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c60b02-1631-4c8e-9921-60ed555de267 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:30.750590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c60b02-1631-4c8e-9921-60ed555de267 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:30.752171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c60b02-1631-4c8e-9921-60ed555de267 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:30.762486 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 432/877] 10.222.0.52 () {56 vars in 1122 bytes} [Thu May 19 13:41:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:30.764158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc66d6-b33c-4230-b114-c4978795e578 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:30.878225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc66d6-b33c-4230-b114-c4978795e578 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:30.918834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2272d6-ee16-4e51-9f6c-5203da919c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:30.920003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2272d6-ee16-4e51-9f6c-5203da919c39 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:30.920482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2272d6-ee16-4e51-9f6c-5203da919c39 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:30.920956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2272d6-ee16-4e51-9f6c-5203da919c39 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:30.921847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2272d6-ee16-4e51-9f6c-5203da919c39 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:30.922332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2272d6-ee16-4e51-9f6c-5203da919c39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:30.923906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2272d6-ee16-4e51-9f6c-5203da919c39 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:30.934995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 446/878] 10.222.0.52 () {56 vars in 1122 bytes} [Thu May 19 13:41:30 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:31.085603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b99a3b-2a92-452d-b79f-3eea5d861d89 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:31.086116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b99a3b-2a92-452d-b79f-3eea5d861d89 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:31.086280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b99a3b-2a92-452d-b79f-3eea5d861d89 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:31.088698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 433/879] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:31.094694 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b646057-150f-48d7-8bd4-56709d7db121 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:31.095012 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b646057-150f-48d7-8bd4-56709d7db121 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:31.095299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b646057-150f-48d7-8bd4-56709d7db121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:31.116858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-8b646057-150f-48d7-8bd4-56709d7db121 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:31.118061 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8b646057-150f-48d7-8bd4-56709d7db121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:31.166561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:31.192984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 447/880] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:31 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:31.215418 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b99a3b-2a92-452d-b79f-3eea5d861d89 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:31.281762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:31.323846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b99a3b-2a92-452d-b79f-3eea5d861d89 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:31.359360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd263f8c-7e4e-4dc7-b683-b7912864b1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:31.359953 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd263f8c-7e4e-4dc7-b683-b7912864b1d2 None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:31.360236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd263f8c-7e4e-4dc7-b683-b7912864b1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:31.360511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd263f8c-7e4e-4dc7-b683-b7912864b1d2 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:31.361236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd263f8c-7e4e-4dc7-b683-b7912864b1d2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:31.363378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd263f8c-7e4e-4dc7-b683-b7912864b1d2 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:31.383217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 434/881] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:41:31 2022] POST /identity/v3/services => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:31.974130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357012cb-9150-4010-b94e-b0c78bba2c12 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:31.974684 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357012cb-9150-4010-b94e-b0c78bba2c12 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:31.974684 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357012cb-9150-4010-b94e-b0c78bba2c12 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:31.976303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 448/882] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:31.981430 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbede33d-5155-4c6d-9ab9-ef59062f95ae None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:31.981974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbede33d-5155-4c6d-9ab9-ef59062f95ae None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:31.982450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbede33d-5155-4c6d-9ab9-ef59062f95ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:32.000112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-cbede33d-5155-4c6d-9ab9-ef59062f95ae None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:32.001577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-cbede33d-5155-4c6d-9ab9-ef59062f95ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:32.079130 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 435/883] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:32.109081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357012cb-9150-4010-b94e-b0c78bba2c12 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:32.223273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357012cb-9150-4010-b94e-b0c78bba2c12 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:32.259433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f80433-867a-40a6-9f33-6cc53abc4095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:32.260160 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f80433-867a-40a6-9f33-6cc53abc4095 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:32.260444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f80433-867a-40a6-9f33-6cc53abc4095 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:32.260721 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f80433-867a-40a6-9f33-6cc53abc4095 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:32.261374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f80433-867a-40a6-9f33-6cc53abc4095 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:32.262749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f80433-867a-40a6-9f33-6cc53abc4095 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-b2f80433-867a-40a6-9f33-6cc53abc4095 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:32.275548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:32.276556 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:32.277387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 449/884] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:41:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:32.285410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbede33d-5155-4c6d-9ab9-ef59062f95ae None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:32.343625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:32.392397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbede33d-5155-4c6d-9ab9-ef59062f95ae None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:32.434812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcdc29-f8ad-45e9-ba98-1c5cf8fe05f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:32.435508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcdc29-f8ad-45e9-ba98-1c5cf8fe05f1 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:32.435791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcdc29-f8ad-45e9-ba98-1c5cf8fe05f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:32.436070 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcdc29-f8ad-45e9-ba98-1c5cf8fe05f1 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:32.436777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bcdc29-f8ad-45e9-ba98-1c5cf8fe05f1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:32.437082 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bcdc29-f8ad-45e9-ba98-1c5cf8fe05f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:32.438442 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bcdc29-f8ad-45e9-ba98-1c5cf8fe05f1 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:32.443672 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 436/885] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:41:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:32.449080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f80433-867a-40a6-9f33-6cc53abc4095 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:32.495531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:32.559271 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f80433-867a-40a6-9f33-6cc53abc4095 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:32.597141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc7a05-6965-482b-b9a3-2420c1a6ec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:32.597809 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc7a05-6965-482b-b9a3-2420c1a6ec01 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:32.598095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc7a05-6965-482b-b9a3-2420c1a6ec01 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:32.598374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc7a05-6965-482b-b9a3-2420c1a6ec01 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:32.599122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc7a05-6965-482b-b9a3-2420c1a6ec01 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:32.599433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc7a05-6965-482b-b9a3-2420c1a6ec01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:32.600806 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc7a05-6965-482b-b9a3-2420c1a6ec01 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:32.606255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 450/886] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:41:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:32.612457 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcdc29-f8ad-45e9-ba98-1c5cf8fe05f1 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:32.725001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcdc29-f8ad-45e9-ba98-1c5cf8fe05f1 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:32.739284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:32.762250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ddfc24-47c5-47bb-a6f6-c288ce53ce54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:32.762929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddfc24-47c5-47bb-a6f6-c288ce53ce54 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:32.763219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddfc24-47c5-47bb-a6f6-c288ce53ce54 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:32.763495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddfc24-47c5-47bb-a6f6-c288ce53ce54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:32.764347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ddfc24-47c5-47bb-a6f6-c288ce53ce54 None admin] RBAC: Adding query filter params (interface=public, service_id=911da5b799f645da9660d9cb65cc5ef6, region_id=RegionOne) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:32.764655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ddfc24-47c5-47bb-a6f6-c288ce53ce54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:32.766065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ddfc24-47c5-47bb-a6f6-c288ce53ce54 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:32.772296 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 437/887] 10.222.0.52 () {56 vars in 1251 bytes} [Thu May 19 13:41:32 2022] GET /identity/v3/endpoints?service_id=911da5b799f645da9660d9cb65cc5ef6&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:32.777651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc7a05-6965-482b-b9a3-2420c1a6ec01 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:32.845737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea47414-5fe1-4f50-ad7f-c0b6341916a1 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:32.846019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea47414-5fe1-4f50-ad7f-c0b6341916a1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:32.846255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea47414-5fe1-4f50-ad7f-c0b6341916a1 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:32.847984 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 438/888] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:32.853064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478263e6-766a-4b0c-a46c-d881d770c365 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:32.853358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478263e6-766a-4b0c-a46c-d881d770c365 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:32.853640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478263e6-766a-4b0c-a46c-d881d770c365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:32.871932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-478263e6-766a-4b0c-a46c-d881d770c365 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:32.873143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-478263e6-766a-4b0c-a46c-d881d770c365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:32.887247 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc7a05-6965-482b-b9a3-2420c1a6ec01 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:32.895620 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:32.928525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366eb80-87e2-48f7-b743-aa6bc2d5752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:32.929194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366eb80-87e2-48f7-b743-aa6bc2d5752b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:32.929482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366eb80-87e2-48f7-b743-aa6bc2d5752b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:32.929761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366eb80-87e2-48f7-b743-aa6bc2d5752b None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:32.930443 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366eb80-87e2-48f7-b743-aa6bc2d5752b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:32.931841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366eb80-87e2-48f7-b743-aa6bc2d5752b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:32.940908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 451/889] 10.222.0.52 () {56 vars in 1087 bytes} [Thu May 19 13:41:32 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:41:32.965363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 439/890] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:32.989211 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366eb80-87e2-48f7-b743-aa6bc2d5752b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:33.112130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366eb80-87e2-48f7-b743-aa6bc2d5752b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:33.152859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096a1ad-e373-405a-b7b3-e8a7e05c730f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:33.153696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096a1ad-e373-405a-b7b3-e8a7e05c730f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:33.153985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096a1ad-e373-405a-b7b3-e8a7e05c730f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:33.154312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096a1ad-e373-405a-b7b3-e8a7e05c730f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:33.167188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096a1ad-e373-405a-b7b3-e8a7e05c730f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-0096a1ad-e373-405a-b7b3-e8a7e05c730f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:33.176153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:33.177177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:33.178015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:33.178015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:33.178015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:33.178015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:33.178015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:33.178015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:33.178015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:33.178015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:33.178015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:33.178015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:33.178015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:41:33.178015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:33.178015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 452/891] 10.222.0.52 () {56 vars in 1120 bytes} [Thu May 19 13:41:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:33.183251 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478263e6-766a-4b0c-a46c-d881d770c365 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:33.287754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478263e6-766a-4b0c-a46c-d881d770c365 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:33.295473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:33.324394 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df752a5-76c8-474c-9bba-a2ff1e3813b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:33.325065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df752a5-76c8-474c-9bba-a2ff1e3813b8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:33.325352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df752a5-76c8-474c-9bba-a2ff1e3813b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:33.325726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df752a5-76c8-474c-9bba-a2ff1e3813b8 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:33.326480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df752a5-76c8-474c-9bba-a2ff1e3813b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:33.326785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df752a5-76c8-474c-9bba-a2ff1e3813b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:33.328180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df752a5-76c8-474c-9bba-a2ff1e3813b8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:33.336989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 440/892] 10.222.0.52 () {56 vars in 1115 bytes} [Thu May 19 13:41:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:33.342464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1a16a-8dd5-4817-acb9-577601ce813e None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:33.342763 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1a16a-8dd5-4817-acb9-577601ce813e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:33.342910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df752a5-76c8-474c-9bba-a2ff1e3813b8 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:33.343095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1a16a-8dd5-4817-acb9-577601ce813e None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:33.344805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 453/893] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:33.350531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cdea4-6e79-4538-b08f-edc105e23d31 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:33.350822 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cdea4-6e79-4538-b08f-edc105e23d31 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:33.351150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cdea4-6e79-4538-b08f-edc105e23d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:33.371493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-783cdea4-6e79-4538-b08f-edc105e23d31 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:33.373469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-783cdea4-6e79-4538-b08f-edc105e23d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:33.416447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:33.463405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df752a5-76c8-474c-9bba-a2ff1e3813b8 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:33.475184 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 454/894] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:33.497346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cdea4-6e79-4538-b08f-edc105e23d31 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:33.506761 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b832b09-5ca1-401f-b4bd-d657c9d67e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:33.508154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b832b09-5ca1-401f-b4bd-d657c9d67e79 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:33.508612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b832b09-5ca1-401f-b4bd-d657c9d67e79 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:33.509034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b832b09-5ca1-401f-b4bd-d657c9d67e79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:33.523920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b832b09-5ca1-401f-b4bd-d657c9d67e79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:33.526277 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b832b09-5ca1-401f-b4bd-d657c9d67e79 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-5b832b09-5ca1-401f-b4bd-d657c9d67e79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:33.550200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:33.551390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:33.552499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:33.552499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:33.552499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:33.552499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:33.554875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 441/895] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:33.560166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b832b09-5ca1-401f-b4bd-d657c9d67e79 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:33.589687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:33.603751 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cdea4-6e79-4538-b08f-edc105e23d31 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:33.644563 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355290c-0cf0-4866-8604-f2d03c67a2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:33.645300 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355290c-0cf0-4866-8604-f2d03c67a2f2 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:33.645691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355290c-0cf0-4866-8604-f2d03c67a2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:33.646074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355290c-0cf0-4866-8604-f2d03c67a2f2 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:33.646962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355290c-0cf0-4866-8604-f2d03c67a2f2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:33.648461 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355290c-0cf0-4866-8604-f2d03c67a2f2 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:33.667314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 455/896] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:41:33 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:41:33.695808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b832b09-5ca1-401f-b4bd-d657c9d67e79 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:33.743022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aee42d-2a13-4246-8218-78122416cdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:33.744064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aee42d-2a13-4246-8218-78122416cdf0 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:33.744513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aee42d-2a13-4246-8218-78122416cdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:33.744954 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aee42d-2a13-4246-8218-78122416cdf0 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:33.745840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aee42d-2a13-4246-8218-78122416cdf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:33.746309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aee42d-2a13-4246-8218-78122416cdf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:33.748141 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aee42d-2a13-4246-8218-78122416cdf0 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:33.763334 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 442/897] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:33.771431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355290c-0cf0-4866-8604-f2d03c67a2f2 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:33.794470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:33.804272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1d8ff-c854-42a0-a217-44a4a56ff592 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:33.804559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1d8ff-c854-42a0-a217-44a4a56ff592 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:33.804837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1d8ff-c854-42a0-a217-44a4a56ff592 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:33.806506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 443/898] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:33.812098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7be71-0e2c-4763-84b3-5c9266f7a266 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:33.812390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7be71-0e2c-4763-84b3-5c9266f7a266 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:33.812672 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7be71-0e2c-4763-84b3-5c9266f7a266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:33.830499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-37b7be71-0e2c-4763-84b3-5c9266f7a266 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:33.832268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-37b7be71-0e2c-4763-84b3-5c9266f7a266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:33.901963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355290c-0cf0-4866-8604-f2d03c67a2f2 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:33.927375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 444/899] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:33.946484 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40ccf3-6a1d-41b8-84fd-cd75c706c92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:33.950960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40ccf3-6a1d-41b8-84fd-cd75c706c92d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:33.950960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40ccf3-6a1d-41b8-84fd-cd75c706c92d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:33.950960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40ccf3-6a1d-41b8-84fd-cd75c706c92d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:33.953701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7be71-0e2c-4763-84b3-5c9266f7a266 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:33.961455 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40ccf3-6a1d-41b8-84fd-cd75c706c92d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:33.961815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:33.963016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40ccf3-6a1d-41b8-84fd-cd75c706c92d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-7c40ccf3-6a1d-41b8-84fd-cd75c706c92d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:33.979400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:33.980400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:33.981344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:33.981344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:33.981344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:33.981344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:33.981344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:33.981344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:33.981344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:33.981344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:33.981344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:41:33.981344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:33.981344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 456/900] 10.222.0.52 () {56 vars in 1117 bytes} [Thu May 19 13:41:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:33.986254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40ccf3-6a1d-41b8-84fd-cd75c706c92d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:34.066216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7be71-0e2c-4763-84b3-5c9266f7a266 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:34.099353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:34.107578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90883661-13c2-4c38-9944-d2ab5afa712c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:34.108737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90883661-13c2-4c38-9944-d2ab5afa712c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:34.109240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90883661-13c2-4c38-9944-d2ab5afa712c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:34.109665 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90883661-13c2-4c38-9944-d2ab5afa712c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:34.111019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90883661-13c2-4c38-9944-d2ab5afa712c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:34.113176 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90883661-13c2-4c38-9944-d2ab5afa712c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:34.123978 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40ccf3-6a1d-41b8-84fd-cd75c706c92d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-90883661-13c2-4c38-9944-d2ab5afa712c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:34.130896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:34.131886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:41:34.133118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:34.137733 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 445/901] 10.222.0.52 () {56 vars in 1132 bytes} [Thu May 19 13:41:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:34.138671 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90883661-13c2-4c38-9944-d2ab5afa712c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:34.168168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2a9f8-e960-4299-b8b8-f368ede4f692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:34.169243 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2a9f8-e960-4299-b8b8-f368ede4f692 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:34.169841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2a9f8-e960-4299-b8b8-f368ede4f692 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:34.170348 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2a9f8-e960-4299-b8b8-f368ede4f692 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:34.171541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2a9f8-e960-4299-b8b8-f368ede4f692 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:34.172082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2a9f8-e960-4299-b8b8-f368ede4f692 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:34.174036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2a9f8-e960-4299-b8b8-f368ede4f692 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:34.200380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 457/902] 10.222.0.52 () {56 vars in 1149 bytes} [Thu May 19 13:41:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:34.205424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2a9f8-e960-4299-b8b8-f368ede4f692 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:34.227107 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:34.263024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90883661-13c2-4c38-9944-d2ab5afa712c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:34.302215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1cd38-96aa-457a-ac02-737e64d42048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:34.303785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1cd38-96aa-457a-ac02-737e64d42048 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:34.304298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1cd38-96aa-457a-ac02-737e64d42048 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:34.304775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1cd38-96aa-457a-ac02-737e64d42048 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:34.305786 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1cd38-96aa-457a-ac02-737e64d42048 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:34.306376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1cd38-96aa-457a-ac02-737e64d42048 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:34.307975 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1cd38-96aa-457a-ac02-737e64d42048 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:34.313972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 446/903] 10.222.0.52 () {56 vars in 1126 bytes} [Thu May 19 13:41:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:34.322744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1cd38-96aa-457a-ac02-737e64d42048 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:34.325417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2a9f8-e960-4299-b8b8-f368ede4f692 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:34.363160 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed4072-a4ad-4e1e-b622-0f5544953f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:34.363806 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed4072-a4ad-4e1e-b622-0f5544953f1e None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:34.364093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed4072-a4ad-4e1e-b622-0f5544953f1e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:34.364370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed4072-a4ad-4e1e-b622-0f5544953f1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:34.373636 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:34.451606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1cd38-96aa-457a-ac02-737e64d42048 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:34.460381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed4072-a4ad-4e1e-b622-0f5544953f1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:34.462494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed4072-a4ad-4e1e-b622-0f5544953f1e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:34.489186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe39a36-81f2-4e39-b0a1-3a16dde7d1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:34.490132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe39a36-81f2-4e39-b0a1-3a16dde7d1e5 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:34.490256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe39a36-81f2-4e39-b0a1-3a16dde7d1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:34.490490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe39a36-81f2-4e39-b0a1-3a16dde7d1e5 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:34.491269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe39a36-81f2-4e39-b0a1-3a16dde7d1e5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:34.491576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe39a36-81f2-4e39-b0a1-3a16dde7d1e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:34.492982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe39a36-81f2-4e39-b0a1-3a16dde7d1e5 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:34.494430 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:34.501250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 447/904] 10.222.0.52 () {56 vars in 1126 bytes} [Thu May 19 13:41:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:34.507237 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe39a36-81f2-4e39-b0a1-3a16dde7d1e5 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:34.592923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 458/905] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:34 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:41:34.594503 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed4072-a4ad-4e1e-b622-0f5544953f1e None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:34.639367 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:34.646727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe39a36-81f2-4e39-b0a1-3a16dde7d1e5 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:34.684993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8dea0-fc46-4fd1-875a-572166f6a040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:34.685681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8dea0-fc46-4fd1-875a-572166f6a040 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:34.685963 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8dea0-fc46-4fd1-875a-572166f6a040 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:34.686235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8dea0-fc46-4fd1-875a-572166f6a040 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:34.687068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8dea0-fc46-4fd1-875a-572166f6a040 None admin] RBAC: Adding query filter params (interface=public, service_id=c8b12e1597924e3ab15b001a47b2e25e, region_id=RegionOne) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:34.687381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8dea0-fc46-4fd1-875a-572166f6a040 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:34.689338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8dea0-fc46-4fd1-875a-572166f6a040 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:34.694904 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 448/906] 10.222.0.52 () {56 vars in 1251 bytes} [Thu May 19 13:41:34 2022] GET /identity/v3/endpoints?service_id=c8b12e1597924e3ab15b001a47b2e25e&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:34.700116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8dea0-fc46-4fd1-875a-572166f6a040 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:34.735057 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed4072-a4ad-4e1e-b622-0f5544953f1e None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:34.774248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:34.776021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10bca3-c926-4f39-b33b-e97a4c5de500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:34.776979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10bca3-c926-4f39-b33b-e97a4c5de500 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:34.777487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10bca3-c926-4f39-b33b-e97a4c5de500 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:34.777905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10bca3-c926-4f39-b33b-e97a4c5de500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:34.791018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10bca3-c926-4f39-b33b-e97a4c5de500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:34.793166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10bca3-c926-4f39-b33b-e97a4c5de500 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-ff10bca3-c926-4f39-b33b-e97a4c5de500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:34.818426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:34.819445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:34.820318 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:34.820318 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:34.820318 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:34.820318 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:34.823033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 459/907] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:34.825570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10bca3-c926-4f39-b33b-e97a4c5de500 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:34.828871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8dea0-fc46-4fd1-875a-572166f6a040 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:34.876421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1cb4d-618f-4ac5-af1b-3ed2749f0801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:34.878618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1cb4d-618f-4ac5-af1b-3ed2749f0801 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:34.879159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1cb4d-618f-4ac5-af1b-3ed2749f0801 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:34.879894 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1cb4d-618f-4ac5-af1b-3ed2749f0801 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:34.881073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1cb4d-618f-4ac5-af1b-3ed2749f0801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:34.883047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1cb4d-618f-4ac5-af1b-3ed2749f0801 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:34.899361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 449/908] 10.222.0.52 () {56 vars in 1087 bytes} [Thu May 19 13:41:34 2022] GET /identity/v3/services => generated 1497 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:41:34.936603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:34.949064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10bca3-c926-4f39-b33b-e97a4c5de500 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:34.989233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed279642-c381-46dc-ba21-315bb0cf009a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:34.989888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed279642-c381-46dc-ba21-315bb0cf009a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:34.990173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed279642-c381-46dc-ba21-315bb0cf009a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:34.990449 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed279642-c381-46dc-ba21-315bb0cf009a None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:34.991280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed279642-c381-46dc-ba21-315bb0cf009a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:34.991590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed279642-c381-46dc-ba21-315bb0cf009a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:34.993000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed279642-c381-46dc-ba21-315bb0cf009a None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:35.008202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 460/909] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:35.020044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1cb4d-618f-4ac5-af1b-3ed2749f0801 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:35.141225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1cb4d-618f-4ac5-af1b-3ed2749f0801 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:35.169638 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:35.180007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb07c5-3e10-4433-abd4-63ecab091a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:35.180798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb07c5-3e10-4433-abd4-63ecab091a77 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:35.181080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb07c5-3e10-4433-abd4-63ecab091a77 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:35.181402 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb07c5-3e10-4433-abd4-63ecab091a77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:35.188436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb07c5-3e10-4433-abd4-63ecab091a77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:35.189915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb07c5-3e10-4433-abd4-63ecab091a77 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-27cb07c5-3e10-4433-abd4-63ecab091a77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:35.204859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:35.205800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:35.207117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:35.207117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:35.207117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:35.207117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:35.207117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:35.207117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:35.207117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:35.207117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:35.207117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 450/910] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:35.212009 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed279642-c381-46dc-ba21-315bb0cf009a None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:35.290474 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8124e85-24aa-48dc-8172-29e46d552471 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:35.290770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8124e85-24aa-48dc-8172-29e46d552471 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:35.290982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8124e85-24aa-48dc-8172-29e46d552471 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:35.292586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 451/911] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:35.297493 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007a24b-2789-486b-90ca-0c04c2f1d187 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:35.297777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007a24b-2789-486b-90ca-0c04c2f1d187 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:35.298061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007a24b-2789-486b-90ca-0c04c2f1d187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:35.316117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-4007a24b-2789-486b-90ca-0c04c2f1d187 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:35.317294 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-4007a24b-2789-486b-90ca-0c04c2f1d187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:35.318526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed279642-c381-46dc-ba21-315bb0cf009a None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:35.353066 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d03ce5-b461-462b-9de8-2f0e693df613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:35.353832 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d03ce5-b461-462b-9de8-2f0e693df613 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:35.354113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d03ce5-b461-462b-9de8-2f0e693df613 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:35.354491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d03ce5-b461-462b-9de8-2f0e693df613 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:35.355313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d03ce5-b461-462b-9de8-2f0e693df613 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:35.355623 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d03ce5-b461-462b-9de8-2f0e693df613 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:35.357061 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d03ce5-b461-462b-9de8-2f0e693df613 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:35.370916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 461/912] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:35.376938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d03ce5-b461-462b-9de8-2f0e693df613 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:35.393404 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:35.405332 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 452/913] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:35.428977 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007a24b-2789-486b-90ca-0c04c2f1d187 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:35.501439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d03ce5-b461-462b-9de8-2f0e693df613 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:35.538523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682bcf0-9015-4733-a241-4326a7538a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:35.539510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682bcf0-9015-4733-a241-4326a7538a46 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:35.539996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682bcf0-9015-4733-a241-4326a7538a46 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:35.540447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682bcf0-9015-4733-a241-4326a7538a46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:35.541503 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682bcf0-9015-4733-a241-4326a7538a46 None admin] RBAC: Adding query filter params (role.id=87751a130d004defb6bc91e40aff384d, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=a989e5551c1640ebbde8768247441627) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:35.541993 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682bcf0-9015-4733-a241-4326a7538a46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:35.543633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682bcf0-9015-4733-a241-4326a7538a46 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:35.557824 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007a24b-2789-486b-90ca-0c04c2f1d187 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:35.558379 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 462/914] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:35 2022] GET /identity/v3/role_assignments?user.id=a989e5551c1640ebbde8768247441627&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=87751a130d004defb6bc91e40aff384d => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:35.579684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:35.600658 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69299f-c2d8-4678-9358-624d7b90586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:35.601726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69299f-c2d8-4678-9358-624d7b90586c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:35.602190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69299f-c2d8-4678-9358-624d7b90586c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:35.602607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69299f-c2d8-4678-9358-624d7b90586c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:35.603480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69299f-c2d8-4678-9358-624d7b90586c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:35.605005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69299f-c2d8-4678-9358-624d7b90586c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-2f69299f-c2d8-4678-9358-624d7b90586c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:35.618426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:35.619766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:41:35.620822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:35.623819 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 453/915] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:41:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:35.630306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682bcf0-9015-4733-a241-4326a7538a46 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:35.716996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:35.746065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682bcf0-9015-4733-a241-4326a7538a46 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:35.781113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc27d0-cb48-4f2b-a7bb-38dac6646308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:35.782024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc27d0-cb48-4f2b-a7bb-38dac6646308 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:35.782450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc27d0-cb48-4f2b-a7bb-38dac6646308 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:35.782885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc27d0-cb48-4f2b-a7bb-38dac6646308 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:35.784389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc27d0-cb48-4f2b-a7bb-38dac6646308 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:35.784891 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc27d0-cb48-4f2b-a7bb-38dac6646308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:35.786398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc27d0-cb48-4f2b-a7bb-38dac6646308 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:35.791997 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 463/916] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:41:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:35.797755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69299f-c2d8-4678-9358-624d7b90586c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:35.912557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69299f-c2d8-4678-9358-624d7b90586c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:35.948120 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88d0ce-c187-47d7-ad2d-f35b75421fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:35.948787 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88d0ce-c187-47d7-ad2d-f35b75421fee None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:35.949074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88d0ce-c187-47d7-ad2d-f35b75421fee None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:35.949352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88d0ce-c187-47d7-ad2d-f35b75421fee None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:35.950061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88d0ce-c187-47d7-ad2d-f35b75421fee None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:35.950422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88d0ce-c187-47d7-ad2d-f35b75421fee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:35.951852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88d0ce-c187-47d7-ad2d-f35b75421fee None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:35.957382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 454/917] 10.222.0.52 () {56 vars in 1114 bytes} [Thu May 19 13:41:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:35.962788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc27d0-cb48-4f2b-a7bb-38dac6646308 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:36.039037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84c5d7-7320-4896-895a-ae8ae5dfdd35 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:36.039366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84c5d7-7320-4896-895a-ae8ae5dfdd35 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:36.039605 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84c5d7-7320-4896-895a-ae8ae5dfdd35 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:36.041228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 455/918] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:36.046261 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558294e-dc26-412d-a53f-f947c864a708 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:36.046493 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558294e-dc26-412d-a53f-f947c864a708 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:36.046776 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558294e-dc26-412d-a53f-f947c864a708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:36.069254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3558294e-dc26-412d-a53f-f947c864a708 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:36.070287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc27d0-cb48-4f2b-a7bb-38dac6646308 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:36.071300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3558294e-dc26-412d-a53f-f947c864a708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:36.112805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd2119-c1cc-48af-a2a3-4d92f5db65f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:36.113517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd2119-c1cc-48af-a2a3-4d92f5db65f5 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:36.113803 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd2119-c1cc-48af-a2a3-4d92f5db65f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:36.114080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd2119-c1cc-48af-a2a3-4d92f5db65f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:36.114838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd2119-c1cc-48af-a2a3-4d92f5db65f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:36.116240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd2119-c1cc-48af-a2a3-4d92f5db65f5 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:36.125106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:36.126332 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:36.145519 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 464/919] 10.222.0.52 () {60 vars in 1144 bytes} [Thu May 19 13:41:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:36.180098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 456/920] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:36.203576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd2119-c1cc-48af-a2a3-4d92f5db65f5 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:36.319485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd2119-c1cc-48af-a2a3-4d92f5db65f5 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:36.356732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442d3ab-39d8-4b25-bc4b-2c2a0e6a2b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:36.357537 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442d3ab-39d8-4b25-bc4b-2c2a0e6a2b72 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:36.357821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442d3ab-39d8-4b25-bc4b-2c2a0e6a2b72 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:36.358099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442d3ab-39d8-4b25-bc4b-2c2a0e6a2b72 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:36.358758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442d3ab-39d8-4b25-bc4b-2c2a0e6a2b72 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:36.360159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442d3ab-39d8-4b25-bc4b-2c2a0e6a2b72 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-9442d3ab-39d8-4b25-bc4b-2c2a0e6a2b72 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:36.373168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:36.374196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:36.375133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 465/921] 10.222.0.52 () {56 vars in 1126 bytes} [Thu May 19 13:41:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:36.380655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558294e-dc26-412d-a53f-f947c864a708 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:36.493657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558294e-dc26-412d-a53f-f947c864a708 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:36.530113 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc555d81-ee6b-42e1-9865-cd7726a8e29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:36.531176 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc555d81-ee6b-42e1-9865-cd7726a8e29f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:36.531693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc555d81-ee6b-42e1-9865-cd7726a8e29f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:36.532162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc555d81-ee6b-42e1-9865-cd7726a8e29f None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:36.533126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc555d81-ee6b-42e1-9865-cd7726a8e29f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:36.533662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc555d81-ee6b-42e1-9865-cd7726a8e29f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:36.535247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc555d81-ee6b-42e1-9865-cd7726a8e29f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:36.540777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 457/922] 10.222.0.52 () {56 vars in 1122 bytes} [Thu May 19 13:41:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:36.547150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442d3ab-39d8-4b25-bc4b-2c2a0e6a2b72 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:36.614106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:36.657117 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442d3ab-39d8-4b25-bc4b-2c2a0e6a2b72 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:36.692398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9ad1c-47c6-4062-b05a-96648c02dca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:36.693068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9ad1c-47c6-4062-b05a-96648c02dca6 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:36.693354 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9ad1c-47c6-4062-b05a-96648c02dca6 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:36.693633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9ad1c-47c6-4062-b05a-96648c02dca6 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:36.694347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9ad1c-47c6-4062-b05a-96648c02dca6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:36.694654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9ad1c-47c6-4062-b05a-96648c02dca6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:36.696071 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9ad1c-47c6-4062-b05a-96648c02dca6 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:36.701324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 466/923] 10.222.0.52 () {56 vars in 1122 bytes} [Thu May 19 13:41:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:36.707837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc555d81-ee6b-42e1-9865-cd7726a8e29f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:36.748068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:36.818263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc555d81-ee6b-42e1-9865-cd7726a8e29f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:36.867690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2cd07-3993-46ff-bbfa-a1586c04da28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:36.868362 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2cd07-3993-46ff-bbfa-a1586c04da28 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:36.868647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2cd07-3993-46ff-bbfa-a1586c04da28 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:36.868923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2cd07-3993-46ff-bbfa-a1586c04da28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:36.869719 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2cd07-3993-46ff-bbfa-a1586c04da28 None admin] RBAC: Adding query filter params (interface=public, service_id=3fa5958cc95a4f0ca500fe9ef7b0999b, region_id=RegionOne) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:36.870033 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2cd07-3993-46ff-bbfa-a1586c04da28 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:36.871432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2cd07-3993-46ff-bbfa-a1586c04da28 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:36.878594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 458/924] 10.222.0.52 () {56 vars in 1251 bytes} [Thu May 19 13:41:36 2022] GET /identity/v3/endpoints?service_id=3fa5958cc95a4f0ca500fe9ef7b0999b&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:36.885011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9ad1c-47c6-4062-b05a-96648c02dca6 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:36.991508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9ad1c-47c6-4062-b05a-96648c02dca6 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:37.015255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:37.028507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c4ac2-4a8e-4cf2-8367-28215cdcaff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:37.029174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c4ac2-4a8e-4cf2-8367-28215cdcaff1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:37.029455 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c4ac2-4a8e-4cf2-8367-28215cdcaff1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:37.029732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c4ac2-4a8e-4cf2-8367-28215cdcaff1 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:37.030416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c4ac2-4a8e-4cf2-8367-28215cdcaff1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:37.031820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c4ac2-4a8e-4cf2-8367-28215cdcaff1 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:37.038591 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 467/925] 10.222.0.52 () {56 vars in 1087 bytes} [Thu May 19 13:41:36 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:41:37.328441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321fec2-84e9-430e-876f-3be5b9894ed0 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:37.328784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321fec2-84e9-430e-876f-3be5b9894ed0 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:37.328905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321fec2-84e9-430e-876f-3be5b9894ed0 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:37.330584 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 459/926] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:37.335876 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae1a41-9a88-47a7-97a5-b52b7903e29a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:37.336325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae1a41-9a88-47a7-97a5-b52b7903e29a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:37.336470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae1a41-9a88-47a7-97a5-b52b7903e29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:37.355140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-5aae1a41-9a88-47a7-97a5-b52b7903e29a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:37.356333 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5aae1a41-9a88-47a7-97a5-b52b7903e29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:37.432482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 468/927] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:37.455985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321fec2-84e9-430e-876f-3be5b9894ed0 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:37.561335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:37.566438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321fec2-84e9-430e-876f-3be5b9894ed0 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:37.603895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5f810-43ef-40a4-b2a0-f2c523711787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:37.604626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5f810-43ef-40a4-b2a0-f2c523711787 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:37.604911 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5f810-43ef-40a4-b2a0-f2c523711787 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:37.605190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5f810-43ef-40a4-b2a0-f2c523711787 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:37.605838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5f810-43ef-40a4-b2a0-f2c523711787 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:37.607293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5f810-43ef-40a4-b2a0-f2c523711787 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-06f5f810-43ef-40a4-b2a0-f2c523711787 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:37.619740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:37.620771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:37.621650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 460/928] 10.222.0.52 () {56 vars in 1132 bytes} [Thu May 19 13:41:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:37.628021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae1a41-9a88-47a7-97a5-b52b7903e29a None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:37.734225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae1a41-9a88-47a7-97a5-b52b7903e29a None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:37.751310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:37.770329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5852c0-4092-4ea6-a220-8e74c99119bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:37.771026 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5852c0-4092-4ea6-a220-8e74c99119bd None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:37.771317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5852c0-4092-4ea6-a220-8e74c99119bd None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:37.771597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5852c0-4092-4ea6-a220-8e74c99119bd None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:37.772312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5852c0-4092-4ea6-a220-8e74c99119bd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:37.772621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5852c0-4092-4ea6-a220-8e74c99119bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:37.774000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5852c0-4092-4ea6-a220-8e74c99119bd None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:37.779222 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 469/929] 10.222.0.52 () {56 vars in 1126 bytes} [Thu May 19 13:41:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:37.785366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5f810-43ef-40a4-b2a0-f2c523711787 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:37.871284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd9e7a-b43d-47b3-9908-802151f74eec None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:37.871597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd9e7a-b43d-47b3-9908-802151f74eec None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:37.872116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd9e7a-b43d-47b3-9908-802151f74eec None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:37.873759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 470/930] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:37.879420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180169b7-353f-4e91-847e-4864b662d112 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:37.879714 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180169b7-353f-4e91-847e-4864b662d112 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:37.880001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180169b7-353f-4e91-847e-4864b662d112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:37.894915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5f810-43ef-40a4-b2a0-f2c523711787 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:37.898764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-180169b7-353f-4e91-847e-4864b662d112 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:37.900000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-180169b7-353f-4e91-847e-4864b662d112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:37.929773 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cac95-24b5-4dcf-94eb-61a1766f14c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:37.930443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cac95-24b5-4dcf-94eb-61a1766f14c0 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:37.930727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cac95-24b5-4dcf-94eb-61a1766f14c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:37.931035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cac95-24b5-4dcf-94eb-61a1766f14c0 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:37.931743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cac95-24b5-4dcf-94eb-61a1766f14c0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:37.932049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cac95-24b5-4dcf-94eb-61a1766f14c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:37.933417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cac95-24b5-4dcf-94eb-61a1766f14c0 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:37.938808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 461/931] 10.222.0.52 () {56 vars in 1126 bytes} [Thu May 19 13:41:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:37.948276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cac95-24b5-4dcf-94eb-61a1766f14c0 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:37.978894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 471/932] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:37.983418 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:37.999096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180169b7-353f-4e91-847e-4864b662d112 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:38.062555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cac95-24b5-4dcf-94eb-61a1766f14c0 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:38.099908 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931e9cd-97ed-4fbf-a7ba-29eca5423507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:38.100840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931e9cd-97ed-4fbf-a7ba-29eca5423507 None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:38.101295 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931e9cd-97ed-4fbf-a7ba-29eca5423507 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:38.101829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931e9cd-97ed-4fbf-a7ba-29eca5423507 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:38.102999 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931e9cd-97ed-4fbf-a7ba-29eca5423507 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:38.104645 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931e9cd-97ed-4fbf-a7ba-29eca5423507 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:38.125065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180169b7-353f-4e91-847e-4864b662d112 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:38.130118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 462/933] 10.222.0.52 () {60 vars in 1144 bytes} [Thu May 19 13:41:37 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:38.138310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:38.175713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef0d5a-3ad4-48b8-b442-f0b2476cd5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:38.176596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef0d5a-3ad4-48b8-b442-f0b2476cd5d3 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:38.176925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef0d5a-3ad4-48b8-b442-f0b2476cd5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:38.177202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef0d5a-3ad4-48b8-b442-f0b2476cd5d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:38.190909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef0d5a-3ad4-48b8-b442-f0b2476cd5d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-faef0d5a-3ad4-48b8-b442-f0b2476cd5d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:38.199772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:38.201003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:38.202044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:38.202044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:38.202044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:38.202044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:38.202044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:38.202044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:38.202044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:38.202044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:38.202044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:38.202044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:38.202044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:41:38.202044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:38.202044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 472/934] 10.222.0.52 () {56 vars in 1120 bytes} [Thu May 19 13:41:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:38.209671 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931e9cd-97ed-4fbf-a7ba-29eca5423507 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:38.312717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:38.335516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931e9cd-97ed-4fbf-a7ba-29eca5423507 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:38.376570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c517c6-f3be-4672-ad39-af7ffcb7ffe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:38.377171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c517c6-f3be-4672-ad39-af7ffcb7ffe0 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:38.377486 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c517c6-f3be-4672-ad39-af7ffcb7ffe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:38.377768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c517c6-f3be-4672-ad39-af7ffcb7ffe0 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:38.378615 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c517c6-f3be-4672-ad39-af7ffcb7ffe0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:38.378961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c517c6-f3be-4672-ad39-af7ffcb7ffe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:38.380336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c517c6-f3be-4672-ad39-af7ffcb7ffe0 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:38.389616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 463/935] 10.222.0.52 () {56 vars in 1115 bytes} [Thu May 19 13:41:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:38.396034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef0d5a-3ad4-48b8-b442-f0b2476cd5d3 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:38.510141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef0d5a-3ad4-48b8-b442-f0b2476cd5d3 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:38.544960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3833c-120c-4a15-87ce-cfcb9d4592fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:38.546154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3833c-120c-4a15-87ce-cfcb9d4592fb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:38.546644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3833c-120c-4a15-87ce-cfcb9d4592fb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:38.547151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3833c-120c-4a15-87ce-cfcb9d4592fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:38.559095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3833c-120c-4a15-87ce-cfcb9d4592fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:38.560823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3833c-120c-4a15-87ce-cfcb9d4592fb None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-ada3833c-120c-4a15-87ce-cfcb9d4592fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:38.579167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:38.580186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:38.581372 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:38.581372 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:38.581372 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:38.581372 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:38.583821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 473/936] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:38.589636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c517c6-f3be-4672-ad39-af7ffcb7ffe0 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:38.649358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c565e7-5080-4da6-b88c-5513834c12fa None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:38.649656 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c565e7-5080-4da6-b88c-5513834c12fa None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:38.649826 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c565e7-5080-4da6-b88c-5513834c12fa None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:38.651476 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 474/937] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:38.656205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e622230-135c-4195-a9cb-c47bae0f4f29 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:38.656492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e622230-135c-4195-a9cb-c47bae0f4f29 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:38.656771 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e622230-135c-4195-a9cb-c47bae0f4f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:38.673792 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-4e622230-135c-4195-a9cb-c47bae0f4f29 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:38.675059 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4e622230-135c-4195-a9cb-c47bae0f4f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:38.690595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:38.701707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c517c6-f3be-4672-ad39-af7ffcb7ffe0 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:38.750890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928d61c-55c1-4ba1-bdd8-967422ca2646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:38.750890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928d61c-55c1-4ba1-bdd8-967422ca2646 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:38.750890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928d61c-55c1-4ba1-bdd8-967422ca2646 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:38.750890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928d61c-55c1-4ba1-bdd8-967422ca2646 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:38.751326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928d61c-55c1-4ba1-bdd8-967422ca2646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:38.751326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928d61c-55c1-4ba1-bdd8-967422ca2646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:38.754885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928d61c-55c1-4ba1-bdd8-967422ca2646 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:38.766923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 464/938] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:38.773511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928d61c-55c1-4ba1-bdd8-967422ca2646 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:38.775802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:38.798885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 475/939] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:38.833104 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e622230-135c-4195-a9cb-c47bae0f4f29 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:38.949190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928d61c-55c1-4ba1-bdd8-967422ca2646 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:38.982632 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e622230-135c-4195-a9cb-c47bae0f4f29 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:39.020742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610003c2-3e0f-433a-a953-1fe5050ad37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:39.022060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610003c2-3e0f-433a-a953-1fe5050ad37a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:39.023397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610003c2-3e0f-433a-a953-1fe5050ad37a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:39.026422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610003c2-3e0f-433a-a953-1fe5050ad37a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:39.038619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610003c2-3e0f-433a-a953-1fe5050ad37a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:39.041043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610003c2-3e0f-433a-a953-1fe5050ad37a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-610003c2-3e0f-433a-a953-1fe5050ad37a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:39.066341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:39.067367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:39.068287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:39.068287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:39.068287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:39.068287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:39.068287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:39.068287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:39.068287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:39.068287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:39.068287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:41:39.068287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:39.070344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c167e2-518f-44d5-a840-0991d1fd5524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:39.072221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c167e2-518f-44d5-a840-0991d1fd5524 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:39.072221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c167e2-518f-44d5-a840-0991d1fd5524 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:39.072221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c167e2-518f-44d5-a840-0991d1fd5524 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:39.072558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 465/940] 10.222.0.52 () {56 vars in 1117 bytes} [Thu May 19 13:41:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:39.072634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c167e2-518f-44d5-a840-0991d1fd5524 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:39.074075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c167e2-518f-44d5-a840-0991d1fd5524 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:39.079037 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: WARNING keystone.server.flask.common [None req-a1c167e2-518f-44d5-a840-0991d1fd5524 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 13:41:39.079196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610003c2-3e0f-433a-a953-1fe5050ad37a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:39.110963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.sql.core [None req-a1c167e2-518f-44d5-a840-0991d1fd5524 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 13:41:39.110963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 13:41:39.110963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [parameters: {'id': '59b6fdf8005141949c971bd705ceba76', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 13:41:39.110963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86630) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 13:41:39.112265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-a1c167e2-518f-44d5-a840-0991d1fd5524 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 59b6fdf8005141949c971bd705ceba76.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 59b6fdf8005141949c971bd705ceba76. May 19 13:41:39.112265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:39.112265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 13:41:39.112265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 13:41:39.112265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 13:41:39.112265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 13:41:39.112265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 13:41:39.112265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application result = self._query(query) May 19 13:41:39.112265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 13:41:39.112265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application conn.query(q) May 19 13:41:39.112265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 13:41:39.112265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 13:41:39.112265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application result.read() May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 13:41:39.113590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application next(self.gen) May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application yield resource May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application next(self.gen) May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.session.rollback() May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.force_reraise() May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise self.value May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application session.commit() May 19 13:41:39.114626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._prepare_impl() May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.session.flush() May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._flush(objects) May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application compat.raise_( May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application flush_context.execute() May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 13:41:39.116045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rec.execute(self) May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 13:41:39.117253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application result = self._query(query) May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application conn.query(q) May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application result.read() May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 13:41:39.118227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [parameters: {'id': '59b6fdf8005141949c971bd705ceba76', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:39.119274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:39.120245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:39.121353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:39.121353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 13:41:39.121353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 13:41:39.121353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:39.121353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:39.121353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 13:41:39.121353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 13:41:39.121353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 59b6fdf8005141949c971bd705ceba76. May 19 13:41:39.121353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:39.121353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 476/941] 10.222.0.52 () {60 vars in 1140 bytes} [Thu May 19 13:41:38 2022] POST /identity/v3/projects => generated 262 bytes in 282 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:39.135978 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c167e2-518f-44d5-a840-0991d1fd5524 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:39.135978 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:39.249364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c167e2-518f-44d5-a840-0991d1fd5524 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:39.255722 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:39.297795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610003c2-3e0f-433a-a953-1fe5050ad37a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:39.302130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7528a-c55b-4da0-a124-b3bf66333bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:39.303406 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7528a-c55b-4da0-a124-b3bf66333bab None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:39.303942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7528a-c55b-4da0-a124-b3bf66333bab None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:39.304420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7528a-c55b-4da0-a124-b3bf66333bab None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:39.316336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7528a-c55b-4da0-a124-b3bf66333bab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:39.319045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7528a-c55b-4da0-a124-b3bf66333bab None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-c5e7528a-c55b-4da0-a124-b3bf66333bab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:39.337364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:39.338305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:39.339226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:39.339226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:39.339226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:39.339226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:39.339226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:39.339226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:39.339226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:39.339226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:39.342999 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 477/942] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:39.345096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7528a-c55b-4da0-a124-b3bf66333bab None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:39.347126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09212aa8-91e6-428f-b740-845bc381a8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:39.348108 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09212aa8-91e6-428f-b740-845bc381a8c8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:39.348909 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09212aa8-91e6-428f-b740-845bc381a8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:39.349384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09212aa8-91e6-428f-b740-845bc381a8c8 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:39.351444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09212aa8-91e6-428f-b740-845bc381a8c8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:39.351992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09212aa8-91e6-428f-b740-845bc381a8c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:39.353620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09212aa8-91e6-428f-b740-845bc381a8c8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:39.377825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 466/943] 10.222.0.52 () {56 vars in 1149 bytes} [Thu May 19 13:41:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:39.383152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09212aa8-91e6-428f-b740-845bc381a8c8 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:39.440754 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:39.492393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7528a-c55b-4da0-a124-b3bf66333bab None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:39.511912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09212aa8-91e6-428f-b740-845bc381a8c8 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:39.529612 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb224f-547b-42d4-a88c-5f7f570b75e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:39.530272 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb224f-547b-42d4-a88c-5f7f570b75e0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:39.530553 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb224f-547b-42d4-a88c-5f7f570b75e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:39.530824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb224f-547b-42d4-a88c-5f7f570b75e0 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:39.531610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb224f-547b-42d4-a88c-5f7f570b75e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:39.531913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb224f-547b-42d4-a88c-5f7f570b75e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:39.533360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb224f-547b-42d4-a88c-5f7f570b75e0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:39.537265 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:39.550352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599d457-9248-4b51-b012-a39ddb267531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:39.551182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599d457-9248-4b51-b012-a39ddb267531 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:39.551469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599d457-9248-4b51-b012-a39ddb267531 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:39.551745 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599d457-9248-4b51-b012-a39ddb267531 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:39.555580 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 478/944] 10.222.0.52 () {56 vars in 1112 bytes} [Thu May 19 13:41:39 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:39.556788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29acd2-71b8-4a2d-9b96-3c145b35b7a7 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:39.557079 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29acd2-71b8-4a2d-9b96-3c145b35b7a7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:39.557357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29acd2-71b8-4a2d-9b96-3c145b35b7a7 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:39.558702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 479/945] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:39.564240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6334f0-1ddb-4c20-b768-c16f4441f7b0 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:39.571624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6334f0-1ddb-4c20-b768-c16f4441f7b0 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:39.571624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6334f0-1ddb-4c20-b768-c16f4441f7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:39.577150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599d457-9248-4b51-b012-a39ddb267531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:39.578705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599d457-9248-4b51-b012-a39ddb267531 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:39.588739 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-3a6334f0-1ddb-4c20-b768-c16f4441f7b0 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:39.590311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3a6334f0-1ddb-4c20-b768-c16f4441f7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-d599d457-9248-4b51-b012-a39ddb267531 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:39.598988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:39.600276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:39.601167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:39.601167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:39.601167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:39.601167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:39.603613 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 467/946] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:39.616692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599d457-9248-4b51-b012-a39ddb267531 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:39.670149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 480/947] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:39.693728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6334f0-1ddb-4c20-b768-c16f4441f7b0 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:39.748265 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599d457-9248-4b51-b012-a39ddb267531 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:39.760272 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:39.788388 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5df020-66db-4780-b289-72f006c1ab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:39.788980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5df020-66db-4780-b289-72f006c1ab78 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:39.789263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5df020-66db-4780-b289-72f006c1ab78 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:39.789536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5df020-66db-4780-b289-72f006c1ab78 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:39.790255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5df020-66db-4780-b289-72f006c1ab78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:39.790563 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5df020-66db-4780-b289-72f006c1ab78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:39.791954 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5df020-66db-4780-b289-72f006c1ab78 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:39.804755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 468/948] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:39.811289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5df020-66db-4780-b289-72f006c1ab78 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:39.816608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6334f0-1ddb-4c20-b768-c16f4441f7b0 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:39.851926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:39.868997 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e98aa-ed83-4049-8d87-f124fbe2e6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:39.869809 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e98aa-ed83-4049-8d87-f124fbe2e6cb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:39.870091 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e98aa-ed83-4049-8d87-f124fbe2e6cb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:39.870367 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e98aa-ed83-4049-8d87-f124fbe2e6cb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:39.871058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e98aa-ed83-4049-8d87-f124fbe2e6cb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:39.872683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e98aa-ed83-4049-8d87-f124fbe2e6cb None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-041e98aa-ed83-4049-8d87-f124fbe2e6cb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:39.885643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:39.887709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:39.889374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 481/949] 10.222.0.52 () {56 vars in 1126 bytes} [Thu May 19 13:41:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:39.897732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e98aa-ed83-4049-8d87-f124fbe2e6cb None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:39.936578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5df020-66db-4780-b289-72f006c1ab78 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:39.972820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093cb616-7b19-4035-90f2-09611ec10279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:39.973796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cb616-7b19-4035-90f2-09611ec10279 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:39.974181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cb616-7b19-4035-90f2-09611ec10279 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:39.974564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093cb616-7b19-4035-90f2-09611ec10279 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:39.982334 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093cb616-7b19-4035-90f2-09611ec10279 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:39.984019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093cb616-7b19-4035-90f2-09611ec10279 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-093cb616-7b19-4035-90f2-09611ec10279 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:40.000690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:40.001779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:40.003457 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:40.003457 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:40.003457 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:40.003457 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:40.003457 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:40.003457 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:40.003457 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:40.003457 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:40.004117 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e98aa-ed83-4049-8d87-f124fbe2e6cb None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:40.005502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 469/950] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:40.011023 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093cb616-7b19-4035-90f2-09611ec10279 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:40.023907 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:40.042185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fa49a-5644-4553-87da-c3f5e1784e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:40.043138 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fa49a-5644-4553-87da-c3f5e1784e51 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:40.043580 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fa49a-5644-4553-87da-c3f5e1784e51 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:40.044051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fa49a-5644-4553-87da-c3f5e1784e51 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:40.044903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77fa49a-5644-4553-87da-c3f5e1784e51 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:40.045382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77fa49a-5644-4553-87da-c3f5e1784e51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:40.046920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77fa49a-5644-4553-87da-c3f5e1784e51 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:40.057646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 482/951] 10.222.0.52 () {56 vars in 1122 bytes} [Thu May 19 13:41:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:40.058781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fa49a-5644-4553-87da-c3f5e1784e51 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:40.122114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:40.131204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093cb616-7b19-4035-90f2-09611ec10279 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:40.166143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fa49a-5644-4553-87da-c3f5e1784e51 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:40.170051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699eea8-9e20-4047-be7e-2e71ec82b2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:40.170844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699eea8-9e20-4047-be7e-2e71ec82b2ac None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:40.171271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699eea8-9e20-4047-be7e-2e71ec82b2ac None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:40.171647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699eea8-9e20-4047-be7e-2e71ec82b2ac None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:40.172544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699eea8-9e20-4047-be7e-2e71ec82b2ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:40.172957 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699eea8-9e20-4047-be7e-2e71ec82b2ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:40.175019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699eea8-9e20-4047-be7e-2e71ec82b2ac None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:40.193573 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 470/952] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:40.194635 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699eea8-9e20-4047-be7e-2e71ec82b2ac None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:40.204081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b07596-b10a-4ccf-958a-13e5df9c6ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:40.204948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b07596-b10a-4ccf-958a-13e5df9c6ff9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:40.205370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b07596-b10a-4ccf-958a-13e5df9c6ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:40.206411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b07596-b10a-4ccf-958a-13e5df9c6ff9 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:40.207523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b07596-b10a-4ccf-958a-13e5df9c6ff9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:40.207980 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b07596-b10a-4ccf-958a-13e5df9c6ff9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:40.209490 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b07596-b10a-4ccf-958a-13e5df9c6ff9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:40.214807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 483/953] 10.222.0.52 () {56 vars in 1122 bytes} [Thu May 19 13:41:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:40.220353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b07596-b10a-4ccf-958a-13e5df9c6ff9 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:40.269573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:40.310632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699eea8-9e20-4047-be7e-2e71ec82b2ac None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:40.326378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b07596-b10a-4ccf-958a-13e5df9c6ff9 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:40.352353 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159296a-7e61-4b30-b3ed-f9c7b3b4d28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:40.353544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159296a-7e61-4b30-b3ed-f9c7b3b4d28f None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:40.353934 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159296a-7e61-4b30-b3ed-f9c7b3b4d28f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:40.354315 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159296a-7e61-4b30-b3ed-f9c7b3b4d28f None admin] PATH_INFO: `/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/a989e5551c1640ebbde8768247441627/roles/87751a130d004defb6bc91e40aff384d` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:40.357458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159296a-7e61-4b30-b3ed-f9c7b3b4d28f None admin] RBAC: Authorizing `identity:create_grant(project_id=2fc6fd42a31942688f570f1ce86be510, user_id=a989e5551c1640ebbde8768247441627, role_id=87751a130d004defb6bc91e40aff384d)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:40.359403 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159296a-7e61-4b30-b3ed-f9c7b3b4d28f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:40.367682 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba16adc-7273-408c-93ad-b504ed2fd3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:40.368857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba16adc-7273-408c-93ad-b504ed2fd3dd None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:40.369295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba16adc-7273-408c-93ad-b504ed2fd3dd None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:40.369721 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba16adc-7273-408c-93ad-b504ed2fd3dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:40.370619 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba16adc-7273-408c-93ad-b504ed2fd3dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:40.372544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba16adc-7273-408c-93ad-b504ed2fd3dd None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:40.382955 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 471/954] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:41:40 2022] PUT /identity/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/a989e5551c1640ebbde8768247441627/roles/87751a130d004defb6bc91e40aff384d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:41:40.407324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 484/955] 10.222.0.52 () {60 vars in 1144 bytes} [Thu May 19 13:41:40 2022] POST /identity/v3/endpoints => generated 349 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:40.536073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2098f4b-94ff-4c17-a044-08f8406b187c None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:40.536515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2098f4b-94ff-4c17-a044-08f8406b187c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:40.536943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2098f4b-94ff-4c17-a044-08f8406b187c None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:40.539463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 472/956] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:40.545818 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37f51f-7924-41dd-b4aa-766c7528423d None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:40.546355 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37f51f-7924-41dd-b4aa-766c7528423d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:40.546654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37f51f-7924-41dd-b4aa-766c7528423d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:40.572266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ef37f51f-7924-41dd-b4aa-766c7528423d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:40.574119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ef37f51f-7924-41dd-b4aa-766c7528423d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:40.664019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 485/957] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:40.688170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2098f4b-94ff-4c17-a044-08f8406b187c None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:40.707422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:40.804907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2098f4b-94ff-4c17-a044-08f8406b187c None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:40.841314 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81faa499-5cb2-48eb-bb91-faae7ef90815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:40.842345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faa499-5cb2-48eb-bb91-faae7ef90815 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:40.842798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faa499-5cb2-48eb-bb91-faae7ef90815 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:40.843264 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faa499-5cb2-48eb-bb91-faae7ef90815 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:40.844062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81faa499-5cb2-48eb-bb91-faae7ef90815 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:40.845594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81faa499-5cb2-48eb-bb91-faae7ef90815 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-81faa499-5cb2-48eb-bb91-faae7ef90815 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:40.859429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:40.860849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:41:40.862754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:40.865632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 473/958] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:40.874933 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef37f51f-7924-41dd-b4aa-766c7528423d None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:40.904644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:40.985791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef37f51f-7924-41dd-b4aa-766c7528423d None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:41.023482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66eedb-c6c9-41a0-9927-2798180d56a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:41.024154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66eedb-c6c9-41a0-9927-2798180d56a5 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:41.024437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66eedb-c6c9-41a0-9927-2798180d56a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:41.024716 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66eedb-c6c9-41a0-9927-2798180d56a5 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:41.025430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66eedb-c6c9-41a0-9927-2798180d56a5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:41.025740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66eedb-c6c9-41a0-9927-2798180d56a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:41.027140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66eedb-c6c9-41a0-9927-2798180d56a5 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:41.037598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 486/959] 10.222.0.52 () {56 vars in 1112 bytes} [Thu May 19 13:41:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:41.038990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81faa499-5cb2-48eb-bb91-faae7ef90815 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:41.123187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:41.147612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81faa499-5cb2-48eb-bb91-faae7ef90815 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:41.185381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3a09d-29ae-4bb7-a061-802b1359d39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:41.186434 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3a09d-29ae-4bb7-a061-802b1359d39e None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:41.186992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3a09d-29ae-4bb7-a061-802b1359d39e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:41.187496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3a09d-29ae-4bb7-a061-802b1359d39e None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:41.188404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3a09d-29ae-4bb7-a061-802b1359d39e None admin] RBAC: Adding query filter params (type=compute) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:41.188916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3a09d-29ae-4bb7-a061-802b1359d39e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:41.191380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3a09d-29ae-4bb7-a061-802b1359d39e None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:41.196859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 474/960] 10.222.0.52 () {56 vars in 1112 bytes} [Thu May 19 13:41:41 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:41.974079 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0e218-54d9-4f51-9aa2-f0ecdc6527e4 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:41.974627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0e218-54d9-4f51-9aa2-f0ecdc6527e4 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:41.974627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0e218-54d9-4f51-9aa2-f0ecdc6527e4 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:41.976198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 487/961] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:41.981690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949e643-bf97-46ee-a030-edb01d66b411 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:41.982084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949e643-bf97-46ee-a030-edb01d66b411 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:41.982289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949e643-bf97-46ee-a030-edb01d66b411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:42.000124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-d949e643-bf97-46ee-a030-edb01d66b411 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:42.001322 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d949e643-bf97-46ee-a030-edb01d66b411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:42.072089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 475/962] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:42.097399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0e218-54d9-4f51-9aa2-f0ecdc6527e4 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:42.200717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:42.218817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0e218-54d9-4f51-9aa2-f0ecdc6527e4 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:42.256094 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaefe0b-9da2-439b-b51a-213c584d35c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:42.257111 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaefe0b-9da2-439b-b51a-213c584d35c0 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:42.257593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaefe0b-9da2-439b-b51a-213c584d35c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:42.258063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaefe0b-9da2-439b-b51a-213c584d35c0 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:42.259003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaefe0b-9da2-439b-b51a-213c584d35c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:42.260648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaefe0b-9da2-439b-b51a-213c584d35c0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:42.275607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.sql.core [None req-cfaefe0b-9da2-439b-b51a-213c584d35c0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 13:41:42.275607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 13:41:42.275607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [parameters: {'user_id': '942e8bc78c25485db67183f5a137ee66', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 13:41:42.275607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86630) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-cfaefe0b-9da2-439b-b51a-213c584d35c0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application result = self._query(query) May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application conn.query(q) May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application result.read() May 19 13:41:42.284726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 13:41:42.285678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application next(self.gen) May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application yield resource May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application next(self.gen) May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.session.rollback() May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.force_reraise() May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise self.value May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application session.commit() May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 13:41:42.286640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._prepare_impl() May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.session.flush() May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._flush(objects) May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application compat.raise_( May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application flush_context.execute() May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rec.execute(self) May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 13:41:42.287578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 13:41:42.288447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application result = self._query(query) May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application conn.query(q) May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application result.read() May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 13:41:42.290110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [parameters: {'user_id': '942e8bc78c25485db67183f5a137ee66', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:42.291852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 13:41:42.293523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 13:41:42.298279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 13:41:42.298279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 13:41:42.298279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 13:41:42.298279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 13:41:42.298279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 13:41:42.298279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:42.303516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 488/963] 10.222.0.52 () {60 vars in 1131 bytes} [Thu May 19 13:41:42 2022] POST /identity/v3/users => generated 160 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:42.307775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949e643-bf97-46ee-a030-edb01d66b411 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:42.387158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:42.420721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949e643-bf97-46ee-a030-edb01d66b411 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:42.456929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabbf18-217c-4a61-a5ea-ca1152b7f2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:42.457734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabbf18-217c-4a61-a5ea-ca1152b7f2a2 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:42.458015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabbf18-217c-4a61-a5ea-ca1152b7f2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:42.458290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabbf18-217c-4a61-a5ea-ca1152b7f2a2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:42.465843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbabbf18-217c-4a61-a5ea-ca1152b7f2a2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:42.467298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbabbf18-217c-4a61-a5ea-ca1152b7f2a2 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-bbabbf18-217c-4a61-a5ea-ca1152b7f2a2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:42.482076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:42.483287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:42.484577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:42.484577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:42.484577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:42.484577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:42.484577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:42.484577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:42.484577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:42.484577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:42.484577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:41:42.484577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:42.484577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 476/964] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:41:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:42.489421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaefe0b-9da2-439b-b51a-213c584d35c0 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:42.596814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaefe0b-9da2-439b-b51a-213c584d35c0 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:42.638658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755b6b7-8946-481e-9890-f73aa8c00b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:42.639382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755b6b7-8946-481e-9890-f73aa8c00b50 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:42.639666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755b6b7-8946-481e-9890-f73aa8c00b50 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:42.639942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755b6b7-8946-481e-9890-f73aa8c00b50 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:42.640778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755b6b7-8946-481e-9890-f73aa8c00b50 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:42.641086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755b6b7-8946-481e-9890-f73aa8c00b50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:42.642809 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755b6b7-8946-481e-9890-f73aa8c00b50 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:42.685414 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 489/965] 10.222.0.52 () {56 vars in 1101 bytes} [Thu May 19 13:41:42 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:42.828453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaa858-4941-40a1-a559-51e3877bf3e4 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:42.828804 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d5d41-2fa1-4432-81f9-4f56bf910458 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:42.828988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaa858-4941-40a1-a559-51e3877bf3e4 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:42.829171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d5d41-2fa1-4432-81f9-4f56bf910458 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:42.829344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaa858-4941-40a1-a559-51e3877bf3e4 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:42.829504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d5d41-2fa1-4432-81f9-4f56bf910458 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:42.831215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 477/967] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:42.831375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 490/967] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:42.836743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3975c8-5999-480f-ab30-a46f6ff474c6 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:42.837035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3975c8-5999-480f-ab30-a46f6ff474c6 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:42.837317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3975c8-5999-480f-ab30-a46f6ff474c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:42.840074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f40054-7ce6-46b1-aa7a-45c8bd9c6faf None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:42.840361 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f40054-7ce6-46b1-aa7a-45c8bd9c6faf None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:42.840648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f40054-7ce6-46b1-aa7a-45c8bd9c6faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:42.856691 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-1b3975c8-5999-480f-ab30-a46f6ff474c6 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:42.858256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1b3975c8-5999-480f-ab30-a46f6ff474c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:42.859564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f6f40054-7ce6-46b1-aa7a-45c8bd9c6faf None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:42.861253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f6f40054-7ce6-46b1-aa7a-45c8bd9c6faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:42.870600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:42.962368 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 478/968] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:42.963056 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 491/969] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:42.979537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3975c8-5999-480f-ab30-a46f6ff474c6 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:42.984592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f40054-7ce6-46b1-aa7a-45c8bd9c6faf None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:43.050581 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:43.093913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f40054-7ce6-46b1-aa7a-45c8bd9c6faf None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:43.095775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3975c8-5999-480f-ab30-a46f6ff474c6 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:43.132054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390eb8b-6bb7-4698-856d-3493b06d68c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:43.132862 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390eb8b-6bb7-4698-856d-3493b06d68c5 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:43.133144 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390eb8b-6bb7-4698-856d-3493b06d68c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:43.133418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390eb8b-6bb7-4698-856d-3493b06d68c5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:43.134000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b8d24-6d6e-4fd8-8b8d-26900805a168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:43.134117 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390eb8b-6bb7-4698-856d-3493b06d68c5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:43.134811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b8d24-6d6e-4fd8-8b8d-26900805a168 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:43.135127 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b8d24-6d6e-4fd8-8b8d-26900805a168 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:43.135406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b8d24-6d6e-4fd8-8b8d-26900805a168 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:43.135513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390eb8b-6bb7-4698-856d-3493b06d68c5 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:43.145841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b8d24-6d6e-4fd8-8b8d-26900805a168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-8390eb8b-6bb7-4698-856d-3493b06d68c5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:43.148567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:43.149548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:43.150380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-9e7b8d24-6d6e-4fd8-8b8d-26900805a168 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:43.153998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:43.161451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:43.162614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:43.162614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:43.162614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:43.162614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:43.162614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:43.162614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:43.162614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:43.162614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:43.162614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:43.162614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:43.162614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:41:43.162614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:43.163342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 492/970] 10.222.0.52 () {56 vars in 1126 bytes} [Thu May 19 13:41:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:43.163417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 479/971] 10.222.0.52 () {56 vars in 1120 bytes} [Thu May 19 13:41:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:43.163486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390eb8b-6bb7-4698-856d-3493b06d68c5 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:43.163565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b8d24-6d6e-4fd8-8b8d-26900805a168 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:43.171048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:43.268742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390eb8b-6bb7-4698-856d-3493b06d68c5 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:43.272303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b8d24-6d6e-4fd8-8b8d-26900805a168 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:43.295506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:43.308630 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e9e33-1427-42ae-ae51-b323968de1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:43.309304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e9e33-1427-42ae-ae51-b323968de1f3 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:43.309589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e9e33-1427-42ae-ae51-b323968de1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:43.309867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e9e33-1427-42ae-ae51-b323968de1f3 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:43.310159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5878e5-ee95-4154-8d06-106476793552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:43.310616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e9e33-1427-42ae-ae51-b323968de1f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:43.310815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5878e5-ee95-4154-8d06-106476793552 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:43.311010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e9e33-1427-42ae-ae51-b323968de1f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:43.311153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5878e5-ee95-4154-8d06-106476793552 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:43.311397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5878e5-ee95-4154-8d06-106476793552 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:43.312094 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5878e5-ee95-4154-8d06-106476793552 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:43.312301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e9e33-1427-42ae-ae51-b323968de1f3 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:43.312404 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5878e5-ee95-4154-8d06-106476793552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:43.313727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5878e5-ee95-4154-8d06-106476793552 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:43.325312 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 480/972] 10.222.0.52 () {56 vars in 1115 bytes} [Thu May 19 13:41:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:43.325473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 493/973] 10.222.0.52 () {56 vars in 1122 bytes} [Thu May 19 13:41:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:43.325575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e9e33-1427-42ae-ae51-b323968de1f3 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:43.326176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5878e5-ee95-4154-8d06-106476793552 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:43.402648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:43.436293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5878e5-ee95-4154-8d06-106476793552 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:43.443159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e9e33-1427-42ae-ae51-b323968de1f3 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:43.474640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891f8b9-0b80-4022-84a9-2f35dae5e228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:43.475257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891f8b9-0b80-4022-84a9-2f35dae5e228 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:43.475535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891f8b9-0b80-4022-84a9-2f35dae5e228 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:43.475822 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891f8b9-0b80-4022-84a9-2f35dae5e228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:43.479953 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea4d95-f4b2-4a56-a262-fa8e274f4ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:43.480703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea4d95-f4b2-4a56-a262-fa8e274f4ad9 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:43.481053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea4d95-f4b2-4a56-a262-fa8e274f4ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:43.481399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea4d95-f4b2-4a56-a262-fa8e274f4ad9 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:43.482173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea4d95-f4b2-4a56-a262-fa8e274f4ad9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:43.482546 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea4d95-f4b2-4a56-a262-fa8e274f4ad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:43.484014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea4d95-f4b2-4a56-a262-fa8e274f4ad9 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:43.487276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891f8b9-0b80-4022-84a9-2f35dae5e228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:43.488804 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891f8b9-0b80-4022-84a9-2f35dae5e228 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:43.489805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 481/974] 10.222.0.52 () {56 vars in 1122 bytes} [Thu May 19 13:41:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:43.495300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea4d95-f4b2-4a56-a262-fa8e274f4ad9 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-8891f8b9-0b80-4022-84a9-2f35dae5e228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:43.509344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:43.510270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:43.511232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:43.511232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:43.511232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:43.511232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:43.511232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 494/975] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:43.515976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891f8b9-0b80-4022-84a9-2f35dae5e228 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:43.551044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:43.603546 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea4d95-f4b2-4a56-a262-fa8e274f4ad9 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:43.616898 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891f8b9-0b80-4022-84a9-2f35dae5e228 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:43.636889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6c65c-5e91-4302-bd36-1944ed2287e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:43.637548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6c65c-5e91-4302-bd36-1944ed2287e5 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:43.637837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6c65c-5e91-4302-bd36-1944ed2287e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:43.638118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6c65c-5e91-4302-bd36-1944ed2287e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:43.638949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6c65c-5e91-4302-bd36-1944ed2287e5 None admin] RBAC: Adding query filter params (interface=admin, service_id=3fa5958cc95a4f0ca500fe9ef7b0999b, region_id=RegionOne) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:43.639267 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6c65c-5e91-4302-bd36-1944ed2287e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:43.640073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:43.640638 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6c65c-5e91-4302-bd36-1944ed2287e5 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:43.646192 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 482/976] 10.222.0.52 () {56 vars in 1249 bytes} [Thu May 19 13:41:43 2022] GET /identity/v3/endpoints?service_id=3fa5958cc95a4f0ca500fe9ef7b0999b&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:43.647557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67227268-08f2-43f5-a699-4d4f3df2d171 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:43.647910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67227268-08f2-43f5-a699-4d4f3df2d171 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:43.648250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67227268-08f2-43f5-a699-4d4f3df2d171 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:43.649534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 483/977] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:43.654308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67227268-08f2-43f5-a699-4d4f3df2d171 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:43.656539 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430af659-5e03-4868-a116-e1b8846efc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:43.657331 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430af659-5e03-4868-a116-e1b8846efc70 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:43.657624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430af659-5e03-4868-a116-e1b8846efc70 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:43.657904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430af659-5e03-4868-a116-e1b8846efc70 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:43.658637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430af659-5e03-4868-a116-e1b8846efc70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:43.659033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430af659-5e03-4868-a116-e1b8846efc70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:43.660399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430af659-5e03-4868-a116-e1b8846efc70 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:43.672383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 495/978] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:43.672383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3f75-98f4-4091-aa4f-12bf0ddd9b72 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:43.672383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3f75-98f4-4091-aa4f-12bf0ddd9b72 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:43.672383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3f75-98f4-4091-aa4f-12bf0ddd9b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:43.691713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f38b3f75-98f4-4091-aa4f-12bf0ddd9b72 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:43.692883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f38b3f75-98f4-4091-aa4f-12bf0ddd9b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:43.756054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67227268-08f2-43f5-a699-4d4f3df2d171 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:43.759611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 496/979] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:43.760419 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b3f75-98f4-4091-aa4f-12bf0ddd9b72 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:43.777948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:43.800603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa2774-700d-46ea-9327-92895299e568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:43.801274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2774-700d-46ea-9327-92895299e568 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:43.801560 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2774-700d-46ea-9327-92895299e568 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:43.801837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2774-700d-46ea-9327-92895299e568 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:43.802550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa2774-700d-46ea-9327-92895299e568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:43.803936 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa2774-700d-46ea-9327-92895299e568 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:43.809710 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 484/980] 10.222.0.52 () {56 vars in 1087 bytes} [Thu May 19 13:41:43 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:41:43.810454 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa2774-700d-46ea-9327-92895299e568 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:43.873133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b3f75-98f4-4091-aa4f-12bf0ddd9b72 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:43.889434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:43.917575 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219c024-d8e9-44b9-a320-908dd9b895af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:43.918313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219c024-d8e9-44b9-a320-908dd9b895af None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:43.918616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219c024-d8e9-44b9-a320-908dd9b895af None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:43.918891 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219c024-d8e9-44b9-a320-908dd9b895af None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:43.927850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa2774-700d-46ea-9327-92895299e568 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:43.928241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219c024-d8e9-44b9-a320-908dd9b895af None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:43.929697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219c024-d8e9-44b9-a320-908dd9b895af None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-3219c024-d8e9-44b9-a320-908dd9b895af None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:43.946890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:43.947956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:43.948953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:43.948953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:43.948953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:43.948953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:43.948953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:43.948953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:43.948953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:43.948953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:43.948953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:41:43.948953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:43.953456 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 497/981] 10.222.0.52 () {56 vars in 1117 bytes} [Thu May 19 13:41:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:43.954284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219c024-d8e9-44b9-a320-908dd9b895af None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:43.973171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29e4a4-e563-4c47-9b66-a30ba557791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:43.974493 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29e4a4-e563-4c47-9b66-a30ba557791e None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:43.975019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29e4a4-e563-4c47-9b66-a30ba557791e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:43.975600 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29e4a4-e563-4c47-9b66-a30ba557791e None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:43.976828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29e4a4-e563-4c47-9b66-a30ba557791e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:43.978409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29e4a4-e563-4c47-9b66-a30ba557791e None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:43.994934 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 485/982] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:41:43 2022] POST /identity/v3/services => generated 242 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:44.078547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219c024-d8e9-44b9-a320-908dd9b895af None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:44.133234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe41115-b3cf-4301-af3a-6351e6805fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:44.133921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe41115-b3cf-4301-af3a-6351e6805fc6 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:44.134205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe41115-b3cf-4301-af3a-6351e6805fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:44.134511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe41115-b3cf-4301-af3a-6351e6805fc6 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:44.135405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe41115-b3cf-4301-af3a-6351e6805fc6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:44.135718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe41115-b3cf-4301-af3a-6351e6805fc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:44.137203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe41115-b3cf-4301-af3a-6351e6805fc6 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:44.162137 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 498/983] 10.222.0.52 () {56 vars in 1149 bytes} [Thu May 19 13:41:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:44.164072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29e4a4-e563-4c47-9b66-a30ba557791e None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:44.204521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:44.276203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29e4a4-e563-4c47-9b66-a30ba557791e None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:44.312427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0cf5d-cbda-4785-b5b7-2070fbc16401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:44.313417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0cf5d-cbda-4785-b5b7-2070fbc16401 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:44.313897 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0cf5d-cbda-4785-b5b7-2070fbc16401 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:44.314356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0cf5d-cbda-4785-b5b7-2070fbc16401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:44.390940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba0cf5d-cbda-4785-b5b7-2070fbc16401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:44.392407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba0cf5d-cbda-4785-b5b7-2070fbc16401 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:44.428662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:44.495316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 486/984] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:44 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:41:44.504104 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe41115-b3cf-4301-af3a-6351e6805fc6 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:44.539378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:44.618564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe41115-b3cf-4301-af3a-6351e6805fc6 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:44.653019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b913254-78ad-44e3-baff-326291237faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:44.653805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b913254-78ad-44e3-baff-326291237faa None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:44.654086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b913254-78ad-44e3-baff-326291237faa None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:44.654357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b913254-78ad-44e3-baff-326291237faa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:44.667582 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b913254-78ad-44e3-baff-326291237faa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:44.669020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b913254-78ad-44e3-baff-326291237faa None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-3b913254-78ad-44e3-baff-326291237faa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:44.686877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:44.687808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:44.688720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:44.688720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:44.688720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:44.688720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:44.688720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 499/985] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:44.693678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0cf5d-cbda-4785-b5b7-2070fbc16401 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:44.797996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0cf5d-cbda-4785-b5b7-2070fbc16401 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:44.832273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32134abc-0426-4dd1-a005-66a61bfab9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:44.832935 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32134abc-0426-4dd1-a005-66a61bfab9f3 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:44.833216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32134abc-0426-4dd1-a005-66a61bfab9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:44.833485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32134abc-0426-4dd1-a005-66a61bfab9f3 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:44.834195 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32134abc-0426-4dd1-a005-66a61bfab9f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:44.834496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32134abc-0426-4dd1-a005-66a61bfab9f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:44.835876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32134abc-0426-4dd1-a005-66a61bfab9f3 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:44.838237 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:44.854213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 487/986] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:44.855383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b913254-78ad-44e3-baff-326291237faa None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:44.933116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:44.955078 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b913254-78ad-44e3-baff-326291237faa None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:44.989133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e33ea2-5bac-41fc-b3d0-a474d8182185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:44.989916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e33ea2-5bac-41fc-b3d0-a474d8182185 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:44.990198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e33ea2-5bac-41fc-b3d0-a474d8182185 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:44.990472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e33ea2-5bac-41fc-b3d0-a474d8182185 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:44.998142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e33ea2-5bac-41fc-b3d0-a474d8182185 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:44.999644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e33ea2-5bac-41fc-b3d0-a474d8182185 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-d3e33ea2-5bac-41fc-b3d0-a474d8182185 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:45.014435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:45.015452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:45.017007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:45.017007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:45.017007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:45.017007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:45.017007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:45.017007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:45.017007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:45.017007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:45.017007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 500/987] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:45.021832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32134abc-0426-4dd1-a005-66a61bfab9f3 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:45.121319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32134abc-0426-4dd1-a005-66a61bfab9f3 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:45.157455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1484d-6b4c-401f-945e-9fa88c0b19d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:45.159165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1484d-6b4c-401f-945e-9fa88c0b19d1 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:45.159375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1484d-6b4c-401f-945e-9fa88c0b19d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:45.159652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1484d-6b4c-401f-945e-9fa88c0b19d1 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:45.160565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1484d-6b4c-401f-945e-9fa88c0b19d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:45.160875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1484d-6b4c-401f-945e-9fa88c0b19d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:45.162345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1484d-6b4c-401f-945e-9fa88c0b19d1 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:45.180287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 488/988] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:45.181505 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e33ea2-5bac-41fc-b3d0-a474d8182185 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:45.281119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e33ea2-5bac-41fc-b3d0-a474d8182185 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:45.315172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5656ca3-fcc1-47e7-991f-0e3d9a7712d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:45.315853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5656ca3-fcc1-47e7-991f-0e3d9a7712d1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:45.316119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5656ca3-fcc1-47e7-991f-0e3d9a7712d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:45.316392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5656ca3-fcc1-47e7-991f-0e3d9a7712d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:45.317208 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5656ca3-fcc1-47e7-991f-0e3d9a7712d1 None admin] RBAC: Adding query filter params (role.id=87751a130d004defb6bc91e40aff384d, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=a989e5551c1640ebbde8768247441627) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:45.317519 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5656ca3-fcc1-47e7-991f-0e3d9a7712d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:45.318959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5656ca3-fcc1-47e7-991f-0e3d9a7712d1 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:45.325019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 501/989] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:45 2022] GET /identity/v3/role_assignments?user.id=a989e5551c1640ebbde8768247441627&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=87751a130d004defb6bc91e40aff384d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:46.138958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325bc4e-4c34-4b2e-bd7c-b8ddd7e79a9f None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:46.139360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325bc4e-4c34-4b2e-bd7c-b8ddd7e79a9f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:46.139568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325bc4e-4c34-4b2e-bd7c-b8ddd7e79a9f None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:46.140963 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 489/990] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:46.148197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41442dd-14e5-4fcf-b9a7-f287b820a685 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:46.148506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41442dd-14e5-4fcf-b9a7-f287b820a685 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:46.148768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41442dd-14e5-4fcf-b9a7-f287b820a685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:46.154260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:46.168420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f41442dd-14e5-4fcf-b9a7-f287b820a685 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:46.169625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f41442dd-14e5-4fcf-b9a7-f287b820a685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:46.238089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 502/991] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:46.261130 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325bc4e-4c34-4b2e-bd7c-b8ddd7e79a9f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:46.310497 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19093e5-9968-4adc-bbd0-a6076a44b1bd None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:46.310807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19093e5-9968-4adc-bbd0-a6076a44b1bd None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:46.311103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19093e5-9968-4adc-bbd0-a6076a44b1bd None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:46.312695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 503/992] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:46.318187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937d6eb-c5e0-41c9-bf5c-e3955b47c7c1 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:46.318478 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937d6eb-c5e0-41c9-bf5c-e3955b47c7c1 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:46.318766 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937d6eb-c5e0-41c9-bf5c-e3955b47c7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:46.321830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:46.339409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-c937d6eb-c5e0-41c9-bf5c-e3955b47c7c1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:46.340666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c937d6eb-c5e0-41c9-bf5c-e3955b47c7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:46.370987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325bc4e-4c34-4b2e-bd7c-b8ddd7e79a9f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:46.411151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e25c4-1745-47bc-8d7f-35900b9abb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:46.411962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e25c4-1745-47bc-8d7f-35900b9abb43 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:46.412246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e25c4-1745-47bc-8d7f-35900b9abb43 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:46.412519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e25c4-1745-47bc-8d7f-35900b9abb43 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:46.413159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e25c4-1745-47bc-8d7f-35900b9abb43 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:46.414512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e25c4-1745-47bc-8d7f-35900b9abb43 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:46.422694 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 504/993] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-ac7e25c4-1745-47bc-8d7f-35900b9abb43 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:46.429515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:46.430521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:46.435995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 490/994] 10.222.0.52 () {56 vars in 1126 bytes} [Thu May 19 13:41:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:46.436838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937d6eb-c5e0-41c9-bf5c-e3955b47c7c1 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:46.448098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e25c4-1745-47bc-8d7f-35900b9abb43 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:46.502544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:46.544988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937d6eb-c5e0-41c9-bf5c-e3955b47c7c1 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:46.560219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e25c4-1745-47bc-8d7f-35900b9abb43 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:46.581103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:46.581676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb6cd3-30c6-4326-b19e-480321d3aa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:46.583051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb6cd3-30c6-4326-b19e-480321d3aa3d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:46.583311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb6cd3-30c6-4326-b19e-480321d3aa3d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:46.583608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb6cd3-30c6-4326-b19e-480321d3aa3d None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:46.584397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb6cd3-30c6-4326-b19e-480321d3aa3d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:46.584705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb6cd3-30c6-4326-b19e-480321d3aa3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:46.586069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb6cd3-30c6-4326-b19e-480321d3aa3d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:46.591523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 505/995] 10.222.0.52 () {56 vars in 1122 bytes} [Thu May 19 13:41:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:46.598504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb6cd3-30c6-4326-b19e-480321d3aa3d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:46.602317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882f422-8f34-4278-a2db-cb8e0c507474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:46.603154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882f422-8f34-4278-a2db-cb8e0c507474 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:46.603435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882f422-8f34-4278-a2db-cb8e0c507474 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:46.603720 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882f422-8f34-4278-a2db-cb8e0c507474 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:46.604354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882f422-8f34-4278-a2db-cb8e0c507474 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:46.605700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882f422-8f34-4278-a2db-cb8e0c507474 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-4882f422-8f34-4278-a2db-cb8e0c507474 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:46.617915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:46.618952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:41:46.619818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:46.624381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 491/996] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:46.625158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882f422-8f34-4278-a2db-cb8e0c507474 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:46.707562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb6cd3-30c6-4326-b19e-480321d3aa3d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:46.733692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882f422-8f34-4278-a2db-cb8e0c507474 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:46.735439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:46.744827 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076facc-ca1b-4677-8025-238f8f0ded86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:46.745491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076facc-ca1b-4677-8025-238f8f0ded86 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:46.745772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076facc-ca1b-4677-8025-238f8f0ded86 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:46.746049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076facc-ca1b-4677-8025-238f8f0ded86 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:46.746749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076facc-ca1b-4677-8025-238f8f0ded86 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:46.747331 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076facc-ca1b-4677-8025-238f8f0ded86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:46.748685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076facc-ca1b-4677-8025-238f8f0ded86 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:46.753639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 506/997] 10.222.0.52 () {56 vars in 1122 bytes} [Thu May 19 13:41:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:46.759264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076facc-ca1b-4677-8025-238f8f0ded86 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:46.779806 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6835dd-8005-4e45-a859-a795be6c1c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:46.781128 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6835dd-8005-4e45-a859-a795be6c1c90 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:46.781902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6835dd-8005-4e45-a859-a795be6c1c90 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:46.782333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6835dd-8005-4e45-a859-a795be6c1c90 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:46.783383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6835dd-8005-4e45-a859-a795be6c1c90 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:46.784755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6835dd-8005-4e45-a859-a795be6c1c90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:46.786711 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6835dd-8005-4e45-a859-a795be6c1c90 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:46.798617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 492/998] 10.222.0.52 () {56 vars in 1112 bytes} [Thu May 19 13:41:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:46.799132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6835dd-8005-4e45-a859-a795be6c1c90 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:46.850919 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:46.868325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076facc-ca1b-4677-8025-238f8f0ded86 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:46.907025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6835dd-8005-4e45-a859-a795be6c1c90 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:46.908393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f246a7-7e21-43b9-a2f2-a970f9b8dbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:46.909076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f246a7-7e21-43b9-a2f2-a970f9b8dbec None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:46.909594 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f246a7-7e21-43b9-a2f2-a970f9b8dbec None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:46.909877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f246a7-7e21-43b9-a2f2-a970f9b8dbec None admin] PATH_INFO: `/v3/endpoints` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:46.911381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f246a7-7e21-43b9-a2f2-a970f9b8dbec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:46.912753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f246a7-7e21-43b9-a2f2-a970f9b8dbec None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:46.928047 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 507/999] 10.222.0.52 () {60 vars in 1144 bytes} [Thu May 19 13:41:46 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:41:46.960893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c309a-0688-474c-ae37-006d284d1b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:46.961488 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c309a-0688-474c-ae37-006d284d1b71 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:46.961774 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c309a-0688-474c-ae37-006d284d1b71 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:46.962049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c309a-0688-474c-ae37-006d284d1b71 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:46.962768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c309a-0688-474c-ae37-006d284d1b71 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:46.963112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c309a-0688-474c-ae37-006d284d1b71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:46.964511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c309a-0688-474c-ae37-006d284d1b71 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:46.970555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 493/1000] 10.222.0.52 () {56 vars in 1112 bytes} [Thu May 19 13:41:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:46.976545 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f246a7-7e21-43b9-a2f2-a970f9b8dbec None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:47.034224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:47.090690 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f246a7-7e21-43b9-a2f2-a970f9b8dbec None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:47.132936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841b376-eb05-4e36-9c5c-1a5abcb767e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:47.133621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841b376-eb05-4e36-9c5c-1a5abcb767e5 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:47.133902 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841b376-eb05-4e36-9c5c-1a5abcb767e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:47.134175 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841b376-eb05-4e36-9c5c-1a5abcb767e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:47.135108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841b376-eb05-4e36-9c5c-1a5abcb767e5 None admin] RBAC: Adding query filter params (interface=public, service_id=c4f55b13d3f34001832c11f9a922cff0, region_id=RegionOne) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:47.135424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841b376-eb05-4e36-9c5c-1a5abcb767e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:47.136805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841b376-eb05-4e36-9c5c-1a5abcb767e5 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:47.142772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 508/1001] 10.222.0.52 () {56 vars in 1251 bytes} [Thu May 19 13:41:46 2022] GET /identity/v3/endpoints?service_id=c4f55b13d3f34001832c11f9a922cff0&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:47.148698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c309a-0688-474c-ae37-006d284d1b71 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:47.256347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c309a-0688-474c-ae37-006d284d1b71 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:47.265273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:47.292639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1f62d-4f56-45cb-8031-d11874e2b86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:47.293312 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1f62d-4f56-45cb-8031-d11874e2b86e None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:47.293602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1f62d-4f56-45cb-8031-d11874e2b86e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:47.293884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1f62d-4f56-45cb-8031-d11874e2b86e None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:47.294567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1f62d-4f56-45cb-8031-d11874e2b86e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:47.295976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1f62d-4f56-45cb-8031-d11874e2b86e None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:47.302561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 494/1002] 10.222.0.52 () {56 vars in 1087 bytes} [Thu May 19 13:41:47 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:41:47.682214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73adf3-d958-45ca-b38c-8b6ffba087d1 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:47.682736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73adf3-d958-45ca-b38c-8b6ffba087d1 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:47.682845 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73adf3-d958-45ca-b38c-8b6ffba087d1 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:47.684329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 509/1003] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:47.689273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2585aea-09b7-4e93-a9fb-4c15703d76c8 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:47.689570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2585aea-09b7-4e93-a9fb-4c15703d76c8 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:47.690509 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2585aea-09b7-4e93-a9fb-4c15703d76c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:47.708304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-d2585aea-09b7-4e93-a9fb-4c15703d76c8 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:47.709501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d2585aea-09b7-4e93-a9fb-4c15703d76c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:47.778375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 495/1004] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:47 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:47.801153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73adf3-d958-45ca-b38c-8b6ffba087d1 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:47.905543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73adf3-d958-45ca-b38c-8b6ffba087d1 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:47.920905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:47.942611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a195e-f258-4fea-b3e3-8164d21dd414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:47.943359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a195e-f258-4fea-b3e3-8164d21dd414 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:47.943643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a195e-f258-4fea-b3e3-8164d21dd414 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:47.943913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a195e-f258-4fea-b3e3-8164d21dd414 None admin] PATH_INFO: `/v3/services/image` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:47.944553 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a195e-f258-4fea-b3e3-8164d21dd414 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:47.945892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a195e-f258-4fea-b3e3-8164d21dd414 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-f43a195e-f258-4fea-b3e3-8164d21dd414 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:47.957588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:47.958568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:47.960090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 510/1005] 10.222.0.52 () {56 vars in 1105 bytes} [Thu May 19 13:41:47 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:47.964603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2585aea-09b7-4e93-a9fb-4c15703d76c8 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:48.064729 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2585aea-09b7-4e93-a9fb-4c15703d76c8 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:48.066363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:48.103330 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daebf0-507e-4739-a402-99039a88a442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:48.103997 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daebf0-507e-4739-a402-99039a88a442 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:48.104386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daebf0-507e-4739-a402-99039a88a442 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:48.104547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daebf0-507e-4739-a402-99039a88a442 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:48.105462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daebf0-507e-4739-a402-99039a88a442 None admin] RBAC: Adding query filter params (name=image) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:48.105724 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daebf0-507e-4739-a402-99039a88a442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:48.107601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daebf0-507e-4739-a402-99039a88a442 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:48.118038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 496/1006] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:48.119538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a195e-f258-4fea-b3e3-8164d21dd414 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:48.223400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a195e-f258-4fea-b3e3-8164d21dd414 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:48.257752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb414f80-b2ab-40c8-afc4-a1dbddcf124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:48.258415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb414f80-b2ab-40c8-afc4-a1dbddcf124d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:48.258695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb414f80-b2ab-40c8-afc4-a1dbddcf124d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:48.258998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb414f80-b2ab-40c8-afc4-a1dbddcf124d None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:48.259697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb414f80-b2ab-40c8-afc4-a1dbddcf124d None admin] RBAC: Adding query filter params (type=image) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:48.260001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb414f80-b2ab-40c8-afc4-a1dbddcf124d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:48.261358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb414f80-b2ab-40c8-afc4-a1dbddcf124d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:48.266085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 511/1007] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:49.210561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13031df6-ed40-4c30-933e-c3ff9664446d None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:49.211156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13031df6-ed40-4c30-933e-c3ff9664446d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:49.211156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13031df6-ed40-4c30-933e-c3ff9664446d None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:49.212762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 497/1008] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:49.218006 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36538aac-1371-478c-a531-7927cf52a1e9 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:49.218422 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36538aac-1371-478c-a531-7927cf52a1e9 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:49.218594 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36538aac-1371-478c-a531-7927cf52a1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:49.235758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-36538aac-1371-478c-a531-7927cf52a1e9 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:49.237673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-36538aac-1371-478c-a531-7927cf52a1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:49.273042 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:49.307133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 512/1009] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:49 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:49.329780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13031df6-ed40-4c30-933e-c3ff9664446d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:49.448421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13031df6-ed40-4c30-933e-c3ff9664446d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:49.483942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93289b-7152-4342-98dd-5753b8e496c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:49.484649 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93289b-7152-4342-98dd-5753b8e496c0 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:49.484927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93289b-7152-4342-98dd-5753b8e496c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:49.485194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93289b-7152-4342-98dd-5753b8e496c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:49.486356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93289b-7152-4342-98dd-5753b8e496c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:49.488677 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93289b-7152-4342-98dd-5753b8e496c0 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:49.490434 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 498/1010] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:49.495859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36538aac-1371-478c-a531-7927cf52a1e9 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:49.551568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec537df-b2bc-4807-a4f2-7230300da31f None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:49.551861 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec537df-b2bc-4807-a4f2-7230300da31f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:49.552138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec537df-b2bc-4807-a4f2-7230300da31f None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:49.553615 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 499/1011] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:49.558240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c1d07-d9e1-471c-bb30-6a0c35a78e68 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:49.558528 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c1d07-d9e1-471c-bb30-6a0c35a78e68 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:49.558803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c1d07-d9e1-471c-bb30-6a0c35a78e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:49.575586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-734c1d07-d9e1-471c-bb30-6a0c35a78e68 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:49.576752 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-734c1d07-d9e1-471c-bb30-6a0c35a78e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:49.597887 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36538aac-1371-478c-a531-7927cf52a1e9 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:49.604593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:49.637199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe79d1c-fdd4-47a4-8d70-56cb4ccf8439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:49.637940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe79d1c-fdd4-47a4-8d70-56cb4ccf8439 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:49.638224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe79d1c-fdd4-47a4-8d70-56cb4ccf8439 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:49.638517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe79d1c-fdd4-47a4-8d70-56cb4ccf8439 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:49.639245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe79d1c-fdd4-47a4-8d70-56cb4ccf8439 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:49.640809 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe79d1c-fdd4-47a4-8d70-56cb4ccf8439 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:49.648789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 500/1012] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:49 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:49.668340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 513/1013] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:41:49 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:49.675101 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c1d07-d9e1-471c-bb30-6a0c35a78e68 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:49.791072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c1d07-d9e1-471c-bb30-6a0c35a78e68 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:49.828198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c8c7c-4be5-4412-aa21-a158b4643011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:49.829012 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c8c7c-4be5-4412-aa21-a158b4643011 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:49.829295 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c8c7c-4be5-4412-aa21-a158b4643011 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:49.829570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c8c7c-4be5-4412-aa21-a158b4643011 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:49.830333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c8c7c-4be5-4412-aa21-a158b4643011 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:49.831807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c8c7c-4be5-4412-aa21-a158b4643011 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-e62c8c7c-4be5-4412-aa21-a158b4643011 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:49.846916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:49.848344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:49.849476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 501/1014] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:49.852764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe79d1c-fdd4-47a4-8d70-56cb4ccf8439 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:49.881983 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:49.963688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe79d1c-fdd4-47a4-8d70-56cb4ccf8439 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:49.998381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a4eef-e2b6-4c7f-8df0-cb2a38d6b067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:49.999086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a4eef-e2b6-4c7f-8df0-cb2a38d6b067 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:49.999373 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a4eef-e2b6-4c7f-8df0-cb2a38d6b067 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:49.999651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a4eef-e2b6-4c7f-8df0-cb2a38d6b067 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:50.000358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a4eef-e2b6-4c7f-8df0-cb2a38d6b067 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:50.000666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a4eef-e2b6-4c7f-8df0-cb2a38d6b067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:50.002102 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a4eef-e2b6-4c7f-8df0-cb2a38d6b067 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:50.007291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 514/1015] 10.222.0.52 () {56 vars in 1112 bytes} [Thu May 19 13:41:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:50.013151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c8c7c-4be5-4412-aa21-a158b4643011 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:50.042592 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:50.116387 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c8c7c-4be5-4412-aa21-a158b4643011 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:50.150982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993f7bc-ee8c-4590-961d-8685c1da4f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:50.151649 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993f7bc-ee8c-4590-961d-8685c1da4f15 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:50.151935 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993f7bc-ee8c-4590-961d-8685c1da4f15 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:50.152209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993f7bc-ee8c-4590-961d-8685c1da4f15 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:50.152911 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993f7bc-ee8c-4590-961d-8685c1da4f15 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:50.153304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993f7bc-ee8c-4590-961d-8685c1da4f15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:50.154665 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993f7bc-ee8c-4590-961d-8685c1da4f15 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:50.164954 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 502/1016] 10.222.0.52 () {56 vars in 1112 bytes} [Thu May 19 13:41:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:50.166049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a4eef-e2b6-4c7f-8df0-cb2a38d6b067 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:50.258535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:50.272656 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a4eef-e2b6-4c7f-8df0-cb2a38d6b067 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:50.307208 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8a05e-055b-40ec-aa5d-16320e686582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:50.307803 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8a05e-055b-40ec-aa5d-16320e686582 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:50.308087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8a05e-055b-40ec-aa5d-16320e686582 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:50.308363 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8a05e-055b-40ec-aa5d-16320e686582 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:50.309116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8a05e-055b-40ec-aa5d-16320e686582 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:50.310609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8a05e-055b-40ec-aa5d-16320e686582 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:50.336338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 515/1017] 10.222.0.52 () {60 vars in 1144 bytes} [Thu May 19 13:41:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:41:50.527225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cebed-723d-4bd7-9c2f-2a529c77426d None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:50.527738 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cebed-723d-4bd7-9c2f-2a529c77426d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:50.527816 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cebed-723d-4bd7-9c2f-2a529c77426d None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:50.529158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 503/1018] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:50.534787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f67dc-dd90-43d0-b28d-97ffe7a11eeb None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:50.535111 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f67dc-dd90-43d0-b28d-97ffe7a11eeb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:50.535394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f67dc-dd90-43d0-b28d-97ffe7a11eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:50.553452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-fb2f67dc-dd90-43d0-b28d-97ffe7a11eeb None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:50.554647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-fb2f67dc-dd90-43d0-b28d-97ffe7a11eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:50.632140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 516/1019] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:50 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:50.654618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cebed-723d-4bd7-9c2f-2a529c77426d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:50.750476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:50.760717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cebed-723d-4bd7-9c2f-2a529c77426d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:50.794692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e9918-f60b-437d-8e2c-747d060b095f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:50.795386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e9918-f60b-437d-8e2c-747d060b095f None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:50.795666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e9918-f60b-437d-8e2c-747d060b095f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:50.795937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e9918-f60b-437d-8e2c-747d060b095f None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:50.796662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e9918-f60b-437d-8e2c-747d060b095f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:50.798014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e9918-f60b-437d-8e2c-747d060b095f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:50.813444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 504/1020] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:41:50 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:51.978903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a03dd-2446-42d3-a30d-c07e8318a9c2 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:51.979438 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a03dd-2446-42d3-a30d-c07e8318a9c2 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:51.979438 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a03dd-2446-42d3-a30d-c07e8318a9c2 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:51.981068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 517/1021] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:51.987522 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda7491-6ff4-4839-a16d-4ef026f7a3f7 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:51.988225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda7491-6ff4-4839-a16d-4ef026f7a3f7 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:51.988225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda7491-6ff4-4839-a16d-4ef026f7a3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:52.008013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-1fda7491-6ff4-4839-a16d-4ef026f7a3f7 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:52.009194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1fda7491-6ff4-4839-a16d-4ef026f7a3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:52.084763 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 505/1022] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:52.109629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a03dd-2446-42d3-a30d-c07e8318a9c2 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:52.181901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:52.215118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a03dd-2446-42d3-a30d-c07e8318a9c2 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:52.249832 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9385f-8ea8-4aba-b1b4-85613d5a9458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:52.250539 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9385f-8ea8-4aba-b1b4-85613d5a9458 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:52.250907 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9385f-8ea8-4aba-b1b4-85613d5a9458 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:52.251174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9385f-8ea8-4aba-b1b4-85613d5a9458 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:52.252367 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9385f-8ea8-4aba-b1b4-85613d5a9458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:52.253972 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9385f-8ea8-4aba-b1b4-85613d5a9458 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:52.255828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 518/1023] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:52.261173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda7491-6ff4-4839-a16d-4ef026f7a3f7 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:52.348523 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:52.373448 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda7491-6ff4-4839-a16d-4ef026f7a3f7 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:52.412791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab8801-d9cc-4d81-8d8e-842b52d13a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:52.413467 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab8801-d9cc-4d81-8d8e-842b52d13a1f None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:52.413747 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab8801-d9cc-4d81-8d8e-842b52d13a1f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:52.414017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab8801-d9cc-4d81-8d8e-842b52d13a1f None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:52.414781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab8801-d9cc-4d81-8d8e-842b52d13a1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:52.416256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab8801-d9cc-4d81-8d8e-842b52d13a1f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:52.438274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 506/1024] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:41:52 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:52.691405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86d60a-838b-4224-9ae9-7a9831bbe736 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:52.691935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86d60a-838b-4224-9ae9-7a9831bbe736 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:52.692359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86d60a-838b-4224-9ae9-7a9831bbe736 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:52.694360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 519/1025] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:52.699505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271ffb1-a97f-4835-a8b5-82b8fa9e1daa None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:52.700038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271ffb1-a97f-4835-a8b5-82b8fa9e1daa None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:52.700533 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271ffb1-a97f-4835-a8b5-82b8fa9e1daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:52.718449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-c271ffb1-a97f-4835-a8b5-82b8fa9e1daa None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:52.719697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c271ffb1-a97f-4835-a8b5-82b8fa9e1daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:52.791040 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 507/1026] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:52.813936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86d60a-838b-4224-9ae9-7a9831bbe736 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:52.917792 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86d60a-838b-4224-9ae9-7a9831bbe736 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:52.957007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013c9fe-7334-4c40-b27e-36f02e7826f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:52.957747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013c9fe-7334-4c40-b27e-36f02e7826f0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:52.958031 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013c9fe-7334-4c40-b27e-36f02e7826f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:52.958309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013c9fe-7334-4c40-b27e-36f02e7826f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:52.965917 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:52.970589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f013c9fe-7334-4c40-b27e-36f02e7826f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-f013c9fe-7334-4c40-b27e-36f02e7826f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:52.979294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:52.980679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:52.982035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:52.982035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:52.982035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:52.982035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:52.982035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:52.982035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:52.982035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:52.982035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:52.982035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:52.982035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:52.982035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:41:52.982035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:52.989061 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 520/1027] 10.222.0.52 () {56 vars in 1120 bytes} [Thu May 19 13:41:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:52.989206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271ffb1-a97f-4835-a8b5-82b8fa9e1daa None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:53.051769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:53.090928 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271ffb1-a97f-4835-a8b5-82b8fa9e1daa None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:53.115397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248a55c-c369-459f-b383-48cdb375028f None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:53.115719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248a55c-c369-459f-b383-48cdb375028f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:53.116002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248a55c-c369-459f-b383-48cdb375028f None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:53.117362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 521/1028] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:53.122457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49011b-feb4-4fb8-bebb-38dd5e5ed807 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:53.122747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49011b-feb4-4fb8-bebb-38dd5e5ed807 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:53.123056 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49011b-feb4-4fb8-bebb-38dd5e5ed807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:53.128042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1fa89-b722-4ed7-aadf-e6d0e7498176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:53.128904 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1fa89-b722-4ed7-aadf-e6d0e7498176 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:53.129193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1fa89-b722-4ed7-aadf-e6d0e7498176 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:53.129465 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1fa89-b722-4ed7-aadf-e6d0e7498176 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:53.130214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1fa89-b722-4ed7-aadf-e6d0e7498176 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:53.130521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1fa89-b722-4ed7-aadf-e6d0e7498176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:53.131923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1fa89-b722-4ed7-aadf-e6d0e7498176 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:53.139736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-fd49011b-feb4-4fb8-bebb-38dd5e5ed807 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:53.144552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 508/1029] 10.222.0.52 () {56 vars in 1115 bytes} [Thu May 19 13:41:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:53.144706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-fd49011b-feb4-4fb8-bebb-38dd5e5ed807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:53.146230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1fa89-b722-4ed7-aadf-e6d0e7498176 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:53.211755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 522/1030] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:53.234959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49011b-feb4-4fb8-bebb-38dd5e5ed807 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:53.258419 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1fa89-b722-4ed7-aadf-e6d0e7498176 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:53.285971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:53.296573 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c92b7b-af64-4cfb-a80b-117f48933469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:53.297363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c92b7b-af64-4cfb-a80b-117f48933469 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:53.297650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c92b7b-af64-4cfb-a80b-117f48933469 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:53.297925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c92b7b-af64-4cfb-a80b-117f48933469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:53.313184 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c92b7b-af64-4cfb-a80b-117f48933469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:53.315664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c92b7b-af64-4cfb-a80b-117f48933469 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-a0c92b7b-af64-4cfb-a80b-117f48933469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:53.334266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:53.335444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:53.336515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:53.336515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:53.336515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:53.336515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:53.336515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 509/1031] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:53.340982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c92b7b-af64-4cfb-a80b-117f48933469 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:53.343195 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49011b-feb4-4fb8-bebb-38dd5e5ed807 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:53.371526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:53.381697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4cbddd-077c-4bb8-8ac9-328fc7d0f58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:53.382489 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cbddd-077c-4bb8-8ac9-328fc7d0f58c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:53.383174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cbddd-077c-4bb8-8ac9-328fc7d0f58c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:53.383485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cbddd-077c-4bb8-8ac9-328fc7d0f58c None admin] PATH_INFO: `/v3/services/image` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:53.384146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4cbddd-077c-4bb8-8ac9-328fc7d0f58c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:53.385909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4cbddd-077c-4bb8-8ac9-328fc7d0f58c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-0c4cbddd-077c-4bb8-8ac9-328fc7d0f58c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:53.398154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:53.399223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:41:53.400070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:53.404480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 523/1032] 10.222.0.52 () {56 vars in 1105 bytes} [Thu May 19 13:41:53 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:53.405323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4cbddd-077c-4bb8-8ac9-328fc7d0f58c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:53.451740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c92b7b-af64-4cfb-a80b-117f48933469 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:53.487712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40d2e9-e1bb-4c1e-a79a-521009278e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:53.488393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40d2e9-e1bb-4c1e-a79a-521009278e2f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:53.488751 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40d2e9-e1bb-4c1e-a79a-521009278e2f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:53.489028 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40d2e9-e1bb-4c1e-a79a-521009278e2f None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:53.489741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e40d2e9-e1bb-4c1e-a79a-521009278e2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:53.490042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e40d2e9-e1bb-4c1e-a79a-521009278e2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:53.491479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e40d2e9-e1bb-4c1e-a79a-521009278e2f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:53.504810 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 510/1033] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:53.510124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40d2e9-e1bb-4c1e-a79a-521009278e2f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:53.513004 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4cbddd-077c-4bb8-8ac9-328fc7d0f58c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:53.546765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999be26a-67ec-4b58-b326-192a70ce8027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:53.547476 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999be26a-67ec-4b58-b326-192a70ce8027 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:53.547759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999be26a-67ec-4b58-b326-192a70ce8027 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:53.548033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999be26a-67ec-4b58-b326-192a70ce8027 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:53.548732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999be26a-67ec-4b58-b326-192a70ce8027 None admin] RBAC: Adding query filter params (name=image) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:53.549712 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999be26a-67ec-4b58-b326-192a70ce8027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:53.550939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999be26a-67ec-4b58-b326-192a70ce8027 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:53.552702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:53.556921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 524/1034] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:53.562469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999be26a-67ec-4b58-b326-192a70ce8027 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:53.619058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40d2e9-e1bb-4c1e-a79a-521009278e2f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:53.625899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:53.656297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45ce0f-5106-43a9-af31-821b32f1f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:53.657074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45ce0f-5106-43a9-af31-821b32f1f5f7 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:53.657456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45ce0f-5106-43a9-af31-821b32f1f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:53.657633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45ce0f-5106-43a9-af31-821b32f1f5f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:53.665443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45ce0f-5106-43a9-af31-821b32f1f5f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:53.666894 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45ce0f-5106-43a9-af31-821b32f1f5f7 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:53.674518 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999be26a-67ec-4b58-b326-192a70ce8027 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-bf45ce0f-5106-43a9-af31-821b32f1f5f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:53.682087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:53.683281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:53.684263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:53.684263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:53.684263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:53.684263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:53.684263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:53.684263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:53.684263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:53.684263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:53.684263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:41:53.684263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:53.684263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 511/1035] 10.222.0.52 () {56 vars in 1093 bytes} [Thu May 19 13:41:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:53.688834 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45ce0f-5106-43a9-af31-821b32f1f5f7 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:53.715286 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b1827-a94b-4359-9450-74a487f42ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:53.715947 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b1827-a94b-4359-9450-74a487f42ea7 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:53.716236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b1827-a94b-4359-9450-74a487f42ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:53.716515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b1827-a94b-4359-9450-74a487f42ea7 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:53.717227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b1827-a94b-4359-9450-74a487f42ea7 None admin] RBAC: Adding query filter params (type=image) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:53.717533 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b1827-a94b-4359-9450-74a487f42ea7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:53.718918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b1827-a94b-4359-9450-74a487f42ea7 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:53.724797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 525/1036] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:53.730411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b1827-a94b-4359-9450-74a487f42ea7 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:53.791073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:53.795037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45ce0f-5106-43a9-af31-821b32f1f5f7 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:53.833234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9838f-ab4c-49b4-81dc-719371cbf045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:53.833903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9838f-ab4c-49b4-81dc-719371cbf045 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:53.834185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9838f-ab4c-49b4-81dc-719371cbf045 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:53.834459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9838f-ab4c-49b4-81dc-719371cbf045 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:53.834659 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b1827-a94b-4359-9450-74a487f42ea7 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:53.835331 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9838f-ab4c-49b4-81dc-719371cbf045 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:53.835636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9838f-ab4c-49b4-81dc-719371cbf045 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:53.837054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9838f-ab4c-49b4-81dc-719371cbf045 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:53.854431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:53.855400 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 512/1037] 10.222.0.52 () {56 vars in 1133 bytes} [Thu May 19 13:41:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:53.861008 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9838f-ab4c-49b4-81dc-719371cbf045 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:53.870647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bc549-5092-4213-b23d-dfd688a44877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:53.871345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bc549-5092-4213-b23d-dfd688a44877 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:53.871629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bc549-5092-4213-b23d-dfd688a44877 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:53.872161 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bc549-5092-4213-b23d-dfd688a44877 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:53.872952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bc549-5092-4213-b23d-dfd688a44877 None admin] RBAC: Adding query filter params (interface=public, service_id=e045140d182b4a6bae96cdbae177179a, region_id=RegionOne) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:53.873268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bc549-5092-4213-b23d-dfd688a44877 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:53.874637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bc549-5092-4213-b23d-dfd688a44877 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:53.884613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 526/1038] 10.222.0.52 () {56 vars in 1251 bytes} [Thu May 19 13:41:53 2022] GET /identity/v3/endpoints?service_id=e045140d182b4a6bae96cdbae177179a&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:53.885557 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bc549-5092-4213-b23d-dfd688a44877 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:53.970406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9838f-ab4c-49b4-81dc-719371cbf045 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:53.989974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bc549-5092-4213-b23d-dfd688a44877 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:54.008508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69291d-d40f-4cd2-807a-da69541ad604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:54.009165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69291d-d40f-4cd2-807a-da69541ad604 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:54.009452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69291d-d40f-4cd2-807a-da69541ad604 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:54.009732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69291d-d40f-4cd2-807a-da69541ad604 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:54.025083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9a715-224d-4484-af54-2178748bbba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:54.025747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9a715-224d-4484-af54-2178748bbba0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:54.026034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9a715-224d-4484-af54-2178748bbba0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:54.026307 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9a715-224d-4484-af54-2178748bbba0 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:54.027029 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9a715-224d-4484-af54-2178748bbba0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:54.028383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9a715-224d-4484-af54-2178748bbba0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:54.035223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 527/1039] 10.222.0.52 () {56 vars in 1087 bytes} [Thu May 19 13:41:53 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:41:54.045323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:54.096699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69291d-d40f-4cd2-807a-da69541ad604 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:54.098182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69291d-d40f-4cd2-807a-da69541ad604 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:54.210904 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 513/1040] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:53 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 13:41:54.213715 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9a715-224d-4484-af54-2178748bbba0 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:54.266071 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:54.321632 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9a715-224d-4484-af54-2178748bbba0 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:54.357784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28d0b-1ca4-49bf-b495-331e4b78d568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:54.358581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28d0b-1ca4-49bf-b495-331e4b78d568 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:54.358948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28d0b-1ca4-49bf-b495-331e4b78d568 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:54.359184 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28d0b-1ca4-49bf-b495-331e4b78d568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:54.370407 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca28d0b-1ca4-49bf-b495-331e4b78d568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:54.372123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca28d0b-1ca4-49bf-b495-331e4b78d568 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-8ca28d0b-1ca4-49bf-b495-331e4b78d568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:54.390074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:54.391342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:54.398274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:54.398274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:54.398274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:54.398274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:54.398274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 528/1041] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:54.400656 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69291d-d40f-4cd2-807a-da69541ad604 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:54.463387 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:54.509164 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69291d-d40f-4cd2-807a-da69541ad604 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:54.544710 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c9443-2477-40c2-b1f8-22ecd4d23403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:54.545395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c9443-2477-40c2-b1f8-22ecd4d23403 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:54.545683 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c9443-2477-40c2-b1f8-22ecd4d23403 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:54.545962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c9443-2477-40c2-b1f8-22ecd4d23403 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:54.546701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c9443-2477-40c2-b1f8-22ecd4d23403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:54.547057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c9443-2477-40c2-b1f8-22ecd4d23403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:54.548466 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c9443-2477-40c2-b1f8-22ecd4d23403 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:54.562918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 514/1042] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:54.568750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28d0b-1ca4-49bf-b495-331e4b78d568 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:54.669775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:54.673411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28d0b-1ca4-49bf-b495-331e4b78d568 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:54.709415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d67bf9-7e49-4758-b7e6-435d71e83776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:54.710217 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d67bf9-7e49-4758-b7e6-435d71e83776 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:54.710501 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d67bf9-7e49-4758-b7e6-435d71e83776 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:54.710777 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d67bf9-7e49-4758-b7e6-435d71e83776 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:54.717774 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d67bf9-7e49-4758-b7e6-435d71e83776 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:54.719249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d67bf9-7e49-4758-b7e6-435d71e83776 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-47d67bf9-7e49-4758-b7e6-435d71e83776 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:54.733437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:54.734359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:54.735657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:54.735657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:54.735657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:54.735657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:54.735657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:54.735657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:54.735657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:54.735657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:54.739576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 529/1043] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:54.740932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c9443-2477-40c2-b1f8-22ecd4d23403 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:54.745446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fddaa2-791a-4e07-99ba-e6022981bb25 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:54.745740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fddaa2-791a-4e07-99ba-e6022981bb25 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:54.746024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fddaa2-791a-4e07-99ba-e6022981bb25 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:54.747657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 530/1044] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:54.753128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd325f-a1b5-403d-9892-04ef794570f0 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:54.753416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd325f-a1b5-403d-9892-04ef794570f0 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:54.753700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd325f-a1b5-403d-9892-04ef794570f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:54.771192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-00cd325f-a1b5-403d-9892-04ef794570f0 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:54.772393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-00cd325f-a1b5-403d-9892-04ef794570f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:54.842708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 531/1045] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:54.856731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c9443-2477-40c2-b1f8-22ecd4d23403 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:54.866430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd325f-a1b5-403d-9892-04ef794570f0 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:54.875942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:54.897196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfda254-3443-4c06-a113-5972ced1d6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:54.897989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfda254-3443-4c06-a113-5972ced1d6c3 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:54.898280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfda254-3443-4c06-a113-5972ced1d6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:54.898534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfda254-3443-4c06-a113-5972ced1d6c3 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:54.899344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfda254-3443-4c06-a113-5972ced1d6c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:54.899652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfda254-3443-4c06-a113-5972ced1d6c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:54.901072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfda254-3443-4c06-a113-5972ced1d6c3 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:54.915145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 515/1046] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:54.921039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfda254-3443-4c06-a113-5972ced1d6c3 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:54.969695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:54.984412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd325f-a1b5-403d-9892-04ef794570f0 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:55.023231 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfda254-3443-4c06-a113-5972ced1d6c3 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:55.031139 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97633f6c-76e0-4d42-be58-e3ae3c906079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:55.032863 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633f6c-76e0-4d42-be58-e3ae3c906079 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:55.033359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633f6c-76e0-4d42-be58-e3ae3c906079 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:55.034664 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633f6c-76e0-4d42-be58-e3ae3c906079 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:55.047337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97633f6c-76e0-4d42-be58-e3ae3c906079 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-97633f6c-76e0-4d42-be58-e3ae3c906079 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:55.054989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:55.055921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:55.057567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:55.057567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:55.057567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:55.057567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:55.057567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:55.057567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:55.057567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:55.057567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:55.057567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:55.057567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:55.057567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:55.057567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:55.057567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 532/1047] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:41:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:55.061733 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a5689-17b1-404b-bc85-7023c9e961fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:55.061860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97633f6c-76e0-4d42-be58-e3ae3c906079 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:55.062384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a5689-17b1-404b-bc85-7023c9e961fc None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:55.062671 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a5689-17b1-404b-bc85-7023c9e961fc None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:55.062981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a5689-17b1-404b-bc85-7023c9e961fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:55.063810 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a5689-17b1-404b-bc85-7023c9e961fc None admin] RBAC: Adding query filter params (role.id=87751a130d004defb6bc91e40aff384d, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=4f6093d0c1fc41d98c58bace81e3d8e5) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:55.064121 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a5689-17b1-404b-bc85-7023c9e961fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:55.066243 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a5689-17b1-404b-bc85-7023c9e961fc None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:55.072022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 516/1048] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:54 2022] GET /identity/v3/role_assignments?user.id=4f6093d0c1fc41d98c58bace81e3d8e5&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=87751a130d004defb6bc91e40aff384d => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:55.176974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97633f6c-76e0-4d42-be58-e3ae3c906079 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:55.213442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9e468-8c2a-4c89-9eff-4681c2f6ac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:55.214465 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e468-8c2a-4c89-9eff-4681c2f6ac4b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:55.214980 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e468-8c2a-4c89-9eff-4681c2f6ac4b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:55.215450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e468-8c2a-4c89-9eff-4681c2f6ac4b None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:55.216403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9e468-8c2a-4c89-9eff-4681c2f6ac4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:55.216896 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9e468-8c2a-4c89-9eff-4681c2f6ac4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:55.218471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9e468-8c2a-4c89-9eff-4681c2f6ac4b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:55.228613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 533/1049] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:41:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:55.234943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a5689-17b1-404b-bc85-7023c9e961fc None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:55.280980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:55.341792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a5689-17b1-404b-bc85-7023c9e961fc None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:55.376900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704a9ec-643e-4106-8ec3-7567a46099cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:55.377956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704a9ec-643e-4106-8ec3-7567a46099cb None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:55.378244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704a9ec-643e-4106-8ec3-7567a46099cb None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:55.378526 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704a9ec-643e-4106-8ec3-7567a46099cb None admin] PATH_INFO: `/v3/users/224011d625e64851b278c7801e304edf` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:55.399173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704a9ec-643e-4106-8ec3-7567a46099cb None admin] RBAC: Authorizing `identity:get_user(user_id=224011d625e64851b278c7801e304edf)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:55.400635 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704a9ec-643e-4106-8ec3-7567a46099cb None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:55.402530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 517/1050] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:41:55 2022] GET /identity/v3/users/224011d625e64851b278c7801e304edf => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:55.408128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9e468-8c2a-4c89-9eff-4681c2f6ac4b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:55.425034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:55.511731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9e468-8c2a-4c89-9eff-4681c2f6ac4b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:55.545644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867066f-3159-4e9a-ae89-8a65ccdeab03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:55.546298 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867066f-3159-4e9a-ae89-8a65ccdeab03 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:55.546586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867066f-3159-4e9a-ae89-8a65ccdeab03 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:55.546935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867066f-3159-4e9a-ae89-8a65ccdeab03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:55.618117 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867066f-3159-4e9a-ae89-8a65ccdeab03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:55.619729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867066f-3159-4e9a-ae89-8a65ccdeab03 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:55.633909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:55.720019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 534/1051] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:55 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:41:55.726349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704a9ec-643e-4106-8ec3-7567a46099cb None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:55.826753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:55.831911 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704a9ec-643e-4106-8ec3-7567a46099cb None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:55.870759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c49756-54fe-42a7-a8b8-c36cd84184d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:55.871671 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c49756-54fe-42a7-a8b8-c36cd84184d5 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:55.872216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c49756-54fe-42a7-a8b8-c36cd84184d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:55.872504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c49756-54fe-42a7-a8b8-c36cd84184d5 None admin] PATH_INFO: `/v3/projects/64fd800f6d0e4e33968655c529cfe669` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:55.873677 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c49756-54fe-42a7-a8b8-c36cd84184d5 None admin] RBAC: Authorizing `identity:get_project(project_id=64fd800f6d0e4e33968655c529cfe669)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:55.875295 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c49756-54fe-42a7-a8b8-c36cd84184d5 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:55.877651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 518/1052] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:41:55 2022] GET /identity/v3/projects/64fd800f6d0e4e33968655c529cfe669 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:55.883616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867066f-3159-4e9a-ae89-8a65ccdeab03 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:55.984793 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867066f-3159-4e9a-ae89-8a65ccdeab03 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:55.998976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:56.021056 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157675c5-344e-4a39-9eef-de064af02104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:56.021713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157675c5-344e-4a39-9eef-de064af02104 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:56.022001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157675c5-344e-4a39-9eef-de064af02104 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:56.022279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157675c5-344e-4a39-9eef-de064af02104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:56.023134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157675c5-344e-4a39-9eef-de064af02104 None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=64fd800f6d0e4e33968655c529cfe669, user.id=224011d625e64851b278c7801e304edf) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:56.023457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157675c5-344e-4a39-9eef-de064af02104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:56.024884 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157675c5-344e-4a39-9eef-de064af02104 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:56.030467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 535/1053] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:41:55 2022] GET /identity/v3/role_assignments?user.id=224011d625e64851b278c7801e304edf&scope.project.id=64fd800f6d0e4e33968655c529cfe669&role.id=0893bb944dd94634accd3906aa391cd2 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:56.287732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239cdf6-dc8c-4c78-97aa-577d77151f6b None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:56.288249 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239cdf6-dc8c-4c78-97aa-577d77151f6b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:56.288358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239cdf6-dc8c-4c78-97aa-577d77151f6b None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:56.289890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 519/1054] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:56.295069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdf392-c4bf-4fe4-b671-1bb2ca2d1eb9 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:56.295452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdf392-c4bf-4fe4-b671-1bb2ca2d1eb9 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:56.295618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdf392-c4bf-4fe4-b671-1bb2ca2d1eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:56.313391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-cebdf392-c4bf-4fe4-b671-1bb2ca2d1eb9 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:56.315207 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-cebdf392-c4bf-4fe4-b671-1bb2ca2d1eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:56.383477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 536/1055] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:56.407021 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239cdf6-dc8c-4c78-97aa-577d77151f6b None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:56.508519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239cdf6-dc8c-4c78-97aa-577d77151f6b None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:56.543952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12706d-5120-4e48-a08e-3721b1cf4870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:56.544653 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12706d-5120-4e48-a08e-3721b1cf4870 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:56.544934 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12706d-5120-4e48-a08e-3721b1cf4870 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:56.545205 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12706d-5120-4e48-a08e-3721b1cf4870 None admin] PATH_INFO: `/v3/services/image` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:56.545844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12706d-5120-4e48-a08e-3721b1cf4870 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:56.547222 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12706d-5120-4e48-a08e-3721b1cf4870 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-7e12706d-5120-4e48-a08e-3721b1cf4870 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:56.559110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:56.560086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:41:56.560948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:56.563182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 520/1056] 10.222.0.52 () {56 vars in 1105 bytes} [Thu May 19 13:41:56 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:56.566789 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdf392-c4bf-4fe4-b671-1bb2ca2d1eb9 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:56.666820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdf392-c4bf-4fe4-b671-1bb2ca2d1eb9 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:56.675428 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:56.703080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf211df3-f278-4964-a425-42d85e491dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:56.704038 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf211df3-f278-4964-a425-42d85e491dc5 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:56.704501 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf211df3-f278-4964-a425-42d85e491dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:56.704982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf211df3-f278-4964-a425-42d85e491dc5 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:56.705895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf211df3-f278-4964-a425-42d85e491dc5 None admin] RBAC: Adding query filter params (name=image) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:56.706381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf211df3-f278-4964-a425-42d85e491dc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:56.707955 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf211df3-f278-4964-a425-42d85e491dc5 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:56.717863 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 537/1057] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:56.718827 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12706d-5120-4e48-a08e-3721b1cf4870 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:56.751591 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:56.827250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12706d-5120-4e48-a08e-3721b1cf4870 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:56.860997 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac375f75-0639-4b1f-9532-d5b6ee3c56c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:56.861807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac375f75-0639-4b1f-9532-d5b6ee3c56c2 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:56.862124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac375f75-0639-4b1f-9532-d5b6ee3c56c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:56.862359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac375f75-0639-4b1f-9532-d5b6ee3c56c2 None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:56.863116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac375f75-0639-4b1f-9532-d5b6ee3c56c2 None admin] RBAC: Adding query filter params (type=image) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:56.863422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac375f75-0639-4b1f-9532-d5b6ee3c56c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:56.864803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac375f75-0639-4b1f-9532-d5b6ee3c56c2 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:56.874730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 521/1058] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:56.875888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf211df3-f278-4964-a425-42d85e491dc5 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:56.975997 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf211df3-f278-4964-a425-42d85e491dc5 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:57.009314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17707086-2a0e-415c-97fe-1562ef2ab83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:57.009964 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17707086-2a0e-415c-97fe-1562ef2ab83d None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:57.010244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17707086-2a0e-415c-97fe-1562ef2ab83d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:57.010515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17707086-2a0e-415c-97fe-1562ef2ab83d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:57.011295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17707086-2a0e-415c-97fe-1562ef2ab83d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:57.012656 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17707086-2a0e-415c-97fe-1562ef2ab83d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:57.032173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 538/1059] 10.222.0.52 () {60 vars in 1144 bytes} [Thu May 19 13:41:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:57.308626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fe455-dcf3-4575-a090-947d50b11e27 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:57.309169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fe455-dcf3-4575-a090-947d50b11e27 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:57.309169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fe455-dcf3-4575-a090-947d50b11e27 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:57.310784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 522/1060] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:41:57.316418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdd45d-f2a0-49ae-bfab-704781228feb None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:57.316842 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdd45d-f2a0-49ae-bfab-704781228feb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:57.317020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdd45d-f2a0-49ae-bfab-704781228feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:57.334643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-a2cdd45d-f2a0-49ae-bfab-704781228feb None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:57.335875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a2cdd45d-f2a0-49ae-bfab-704781228feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:57.357600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:57.418732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 539/1061] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:41:57.442713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53fe455-dcf3-4575-a090-947d50b11e27 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:57.540137 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:57.549458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53fe455-dcf3-4575-a090-947d50b11e27 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:57.586853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e50ddb-7b24-4799-8361-755d7fcb326c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:57.587793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e50ddb-7b24-4799-8361-755d7fcb326c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:57.588078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e50ddb-7b24-4799-8361-755d7fcb326c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:57.588355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e50ddb-7b24-4799-8361-755d7fcb326c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:57.599520 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e50ddb-7b24-4799-8361-755d7fcb326c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-79e50ddb-7b24-4799-8361-755d7fcb326c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:57.607616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:57.608544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:57.609389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:57.609389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:57.609389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:57.609389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:57.609389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:57.609389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:57.609389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:57.609389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:57.609389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:57.609389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:57.609389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:41:57.609389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:57.609389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 523/1062] 10.222.0.52 () {56 vars in 1120 bytes} [Thu May 19 13:41:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:57.616673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdd45d-f2a0-49ae-bfab-704781228feb None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:57.717419 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdd45d-f2a0-49ae-bfab-704781228feb None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:57.752140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1198b-9066-46c0-8062-1fbb88acd49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:57.753017 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1198b-9066-46c0-8062-1fbb88acd49c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:57.753327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1198b-9066-46c0-8062-1fbb88acd49c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:57.753580 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1198b-9066-46c0-8062-1fbb88acd49c None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:57.754333 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1198b-9066-46c0-8062-1fbb88acd49c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:57.754641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1198b-9066-46c0-8062-1fbb88acd49c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:57.756037 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1198b-9066-46c0-8062-1fbb88acd49c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:57.769229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 540/1063] 10.222.0.52 () {56 vars in 1115 bytes} [Thu May 19 13:41:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:57.770681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e50ddb-7b24-4799-8361-755d7fcb326c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:57.878716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e50ddb-7b24-4799-8361-755d7fcb326c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:57.914622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1a855-b181-4aa7-b7f9-95a567e36a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:57.915476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a855-b181-4aa7-b7f9-95a567e36a1b None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:57.915767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a855-b181-4aa7-b7f9-95a567e36a1b None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:57.916043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a855-b181-4aa7-b7f9-95a567e36a1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:57.928224 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1a855-b181-4aa7-b7f9-95a567e36a1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:57.929668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1a855-b181-4aa7-b7f9-95a567e36a1b None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-74d1a855-b181-4aa7-b7f9-95a567e36a1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:57.947757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:57.948632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:57.949543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:57.949543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:57.949543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:57.949543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:57.949543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 524/1064] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:57.954513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1198b-9066-46c0-8062-1fbb88acd49c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:57.971352 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:58.058001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1198b-9066-46c0-8062-1fbb88acd49c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:58.093329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255d67f-e22d-4b66-b90f-4d5c9516dde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:58.094018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255d67f-e22d-4b66-b90f-4d5c9516dde4 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:58.094302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255d67f-e22d-4b66-b90f-4d5c9516dde4 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:58.094579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255d67f-e22d-4b66-b90f-4d5c9516dde4 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:58.095357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255d67f-e22d-4b66-b90f-4d5c9516dde4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:58.096197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255d67f-e22d-4b66-b90f-4d5c9516dde4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:58.097600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255d67f-e22d-4b66-b90f-4d5c9516dde4 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:58.117350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 541/1065] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:58.118841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1a855-b181-4aa7-b7f9-95a567e36a1b None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:58.129103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:58.224747 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1a855-b181-4aa7-b7f9-95a567e36a1b None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:58.259541 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb65f6-a9c1-47ad-b33e-a0021957c6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:58.260336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb65f6-a9c1-47ad-b33e-a0021957c6e0 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:58.260619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb65f6-a9c1-47ad-b33e-a0021957c6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:58.260894 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb65f6-a9c1-47ad-b33e-a0021957c6e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:58.268539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb65f6-a9c1-47ad-b33e-a0021957c6e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:58.269975 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb65f6-a9c1-47ad-b33e-a0021957c6e0 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:58.273235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5603d-cd15-4069-a918-0628d748d88e None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:58.273566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5603d-cd15-4069-a918-0628d748d88e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:58.273872 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5603d-cd15-4069-a918-0628d748d88e None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:58.275365 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 542/1066] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:41:58.281051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afad143-d17c-44bf-aed6-ee29c678d8d9 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:58.281345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afad143-d17c-44bf-aed6-ee29c678d8d9 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:58.281634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afad143-d17c-44bf-aed6-ee29c678d8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-dfcb65f6-a9c1-47ad-b33e-a0021957c6e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:41:58.287272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:58.288208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:58.289160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:58.289160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:58.289160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:41:58.289160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:41:58.289160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:41:58.289160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:41:58.289160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:41:58.289160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:41:58.289160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:41:58.289160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:58.289160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 525/1067] 10.222.0.52 () {56 vars in 1093 bytes} [Thu May 19 13:41:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:58.294195 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb65f6-a9c1-47ad-b33e-a0021957c6e0 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:58.302314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-9afad143-d17c-44bf-aed6-ee29c678d8d9 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:58.303528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9afad143-d17c-44bf-aed6-ee29c678d8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:58.372701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:58.378899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 543/1068] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:58.400094 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afad143-d17c-44bf-aed6-ee29c678d8d9 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:58.404345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:58.417325 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb65f6-a9c1-47ad-b33e-a0021957c6e0 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:58.459406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8148b-eeb7-465c-a9e8-acbe07a66c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:58.460238 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8148b-eeb7-465c-a9e8-acbe07a66c3c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:58.460530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8148b-eeb7-465c-a9e8-acbe07a66c3c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:58.460802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8148b-eeb7-465c-a9e8-acbe07a66c3c None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:58.461651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8148b-eeb7-465c-a9e8-acbe07a66c3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:58.461966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8148b-eeb7-465c-a9e8-acbe07a66c3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:58.463431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8148b-eeb7-465c-a9e8-acbe07a66c3c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:58.481310 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 526/1069] 10.222.0.52 () {56 vars in 1133 bytes} [Thu May 19 13:41:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:58.486640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8148b-eeb7-465c-a9e8-acbe07a66c3c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:58.509853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afad143-d17c-44bf-aed6-ee29c678d8d9 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:58.545584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744baa6-e132-4a7e-a201-3a5dd24e955f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:58.546408 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744baa6-e132-4a7e-a201-3a5dd24e955f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:58.546696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744baa6-e132-4a7e-a201-3a5dd24e955f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:58.547006 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744baa6-e132-4a7e-a201-3a5dd24e955f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:58.557468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744baa6-e132-4a7e-a201-3a5dd24e955f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-c744baa6-e132-4a7e-a201-3a5dd24e955f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:41:58.565132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:58.566073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:58.566960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:58.566960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:58.566960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:58.566960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:58.566960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:41:58.566960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:41:58.566960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:41:58.566960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:41:58.566960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:41:58.566960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:41:58.566960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:41:58.566960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:58.571011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 544/1070] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:41:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:58.572488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744baa6-e132-4a7e-a201-3a5dd24e955f None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:58.591635 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8148b-eeb7-465c-a9e8-acbe07a66c3c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:58.615340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:58.629875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c120d4-58c5-4a79-906e-5c7062ce679f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:58.630697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c120d4-58c5-4a79-906e-5c7062ce679f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:58.631030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c120d4-58c5-4a79-906e-5c7062ce679f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:58.631303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c120d4-58c5-4a79-906e-5c7062ce679f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:58.641959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c120d4-58c5-4a79-906e-5c7062ce679f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:58.643266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:58.643538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c120d4-58c5-4a79-906e-5c7062ce679f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-78c120d4-58c5-4a79-906e-5c7062ce679f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:58.662054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:58.663030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:58.664678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:58.664678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:58.664678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:58.664678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:58.664678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 527/1071] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:58.668664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c120d4-58c5-4a79-906e-5c7062ce679f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:58.688336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744baa6-e132-4a7e-a201-3a5dd24e955f None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:58.724713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50650bf-772e-4a8f-a353-d1c7f053b668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:58.725495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50650bf-772e-4a8f-a353-d1c7f053b668 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:58.725780 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50650bf-772e-4a8f-a353-d1c7f053b668 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:58.726176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50650bf-772e-4a8f-a353-d1c7f053b668 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:58.726943 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50650bf-772e-4a8f-a353-d1c7f053b668 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:58.727252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50650bf-772e-4a8f-a353-d1c7f053b668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:58.728716 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50650bf-772e-4a8f-a353-d1c7f053b668 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:58.738248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 545/1072] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:41:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:58.743895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50650bf-772e-4a8f-a353-d1c7f053b668 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:58.773193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c120d4-58c5-4a79-906e-5c7062ce679f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:58.811651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf472f40-83fb-438d-a7aa-6878c72efe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:58.812372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf472f40-83fb-438d-a7aa-6878c72efe83 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:58.812753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf472f40-83fb-438d-a7aa-6878c72efe83 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:58.813032 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf472f40-83fb-438d-a7aa-6878c72efe83 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:58.813761 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf472f40-83fb-438d-a7aa-6878c72efe83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:58.814071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf472f40-83fb-438d-a7aa-6878c72efe83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:58.816247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf472f40-83fb-438d-a7aa-6878c72efe83 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:58.828548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 528/1073] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:58.834706 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf472f40-83fb-438d-a7aa-6878c72efe83 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:58.856287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50650bf-772e-4a8f-a353-d1c7f053b668 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:58.875219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:58.882291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:58.892704 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0c3e1-a18a-4d0d-a80c-4e15205a57d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:58.893573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0c3e1-a18a-4d0d-a80c-4e15205a57d2 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:58.893862 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0c3e1-a18a-4d0d-a80c-4e15205a57d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:58.894147 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0c3e1-a18a-4d0d-a80c-4e15205a57d2 None admin] PATH_INFO: `/v3/users/224011d625e64851b278c7801e304edf` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:58.895370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0c3e1-a18a-4d0d-a80c-4e15205a57d2 None admin] RBAC: Authorizing `identity:get_user(user_id=224011d625e64851b278c7801e304edf)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:58.896791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0c3e1-a18a-4d0d-a80c-4e15205a57d2 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:58.898575 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 546/1074] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:41:58 2022] GET /identity/v3/users/224011d625e64851b278c7801e304edf => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:58.903860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0c3e1-a18a-4d0d-a80c-4e15205a57d2 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:58.948494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf472f40-83fb-438d-a7aa-6878c72efe83 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:58.984467 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81abf60-c630-41d8-ae1d-5332a64d7bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:58.985655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81abf60-c630-41d8-ae1d-5332a64d7bb4 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:58.986140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81abf60-c630-41d8-ae1d-5332a64d7bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:58.987391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81abf60-c630-41d8-ae1d-5332a64d7bb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:58.994314 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81abf60-c630-41d8-ae1d-5332a64d7bb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:58.995789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81abf60-c630-41d8-ae1d-5332a64d7bb4 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:59.009188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0c3e1-a18a-4d0d-a80c-4e15205a57d2 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-e81abf60-c630-41d8-ae1d-5332a64d7bb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:41:59.010198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:59.011227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:59.012643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:41:59.012643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:41:59.012643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:41:59.012643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:41:59.012643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:41:59.012643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:41:59.012643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:41:59.012643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:41:59.012643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 529/1075] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:41:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:41:59.017060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81abf60-c630-41d8-ae1d-5332a64d7bb4 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:59.049696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4eb1c7-1570-4218-a2f3-4a47665f2b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:59.050549 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eb1c7-1570-4218-a2f3-4a47665f2b48 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:59.050829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eb1c7-1570-4218-a2f3-4a47665f2b48 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:59.051127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eb1c7-1570-4218-a2f3-4a47665f2b48 None admin] PATH_INFO: `/v3/projects/64fd800f6d0e4e33968655c529cfe669` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:59.052316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4eb1c7-1570-4218-a2f3-4a47665f2b48 None admin] RBAC: Authorizing `identity:get_project(project_id=64fd800f6d0e4e33968655c529cfe669)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:59.053738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4eb1c7-1570-4218-a2f3-4a47665f2b48 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:59.055464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 547/1076] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:41:58 2022] GET /identity/v3/projects/64fd800f6d0e4e33968655c529cfe669 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:59.061089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4eb1c7-1570-4218-a2f3-4a47665f2b48 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:59.118548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:59.125571 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:59.126013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81abf60-c630-41d8-ae1d-5332a64d7bb4 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:59.163798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98aa75-f5ab-4dd0-a9a4-fcfe3485c827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:59.164473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98aa75-f5ab-4dd0-a9a4-fcfe3485c827 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:59.164759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98aa75-f5ab-4dd0-a9a4-fcfe3485c827 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:59.165034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98aa75-f5ab-4dd0-a9a4-fcfe3485c827 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:59.165838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98aa75-f5ab-4dd0-a9a4-fcfe3485c827 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:59.166146 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98aa75-f5ab-4dd0-a9a4-fcfe3485c827 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:59.167642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98aa75-f5ab-4dd0-a9a4-fcfe3485c827 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:59.173722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4eb1c7-1570-4218-a2f3-4a47665f2b48 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:59.180447 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 530/1077] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:41:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:41:59.186236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98aa75-f5ab-4dd0-a9a4-fcfe3485c827 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:59.214065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289428ba-0608-4d6b-8f65-acd321d0acb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:59.214923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289428ba-0608-4d6b-8f65-acd321d0acb8 None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:59.215210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289428ba-0608-4d6b-8f65-acd321d0acb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:59.215781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289428ba-0608-4d6b-8f65-acd321d0acb8 None admin] PATH_INFO: `/v3/projects/64fd800f6d0e4e33968655c529cfe669/users/224011d625e64851b278c7801e304edf/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:59.217475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289428ba-0608-4d6b-8f65-acd321d0acb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=64fd800f6d0e4e33968655c529cfe669, user_id=224011d625e64851b278c7801e304edf, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:59.218921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289428ba-0608-4d6b-8f65-acd321d0acb8 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:59.233930 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 548/1078] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:41:59 2022] PUT /identity/v3/projects/64fd800f6d0e4e33968655c529cfe669/users/224011d625e64851b278c7801e304edf/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:41:59.299373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98aa75-f5ab-4dd0-a9a4-fcfe3485c827 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:59.337641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d713a5-1432-4b5d-88e1-fb656b598ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:59.338380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2e456-dd73-4db3-9be7-c511a1fa7549 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:59.338563 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d713a5-1432-4b5d-88e1-fb656b598ae2 None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:59.338689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2e456-dd73-4db3-9be7-c511a1fa7549 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:59.338846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d713a5-1432-4b5d-88e1-fb656b598ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:59.338972 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2e456-dd73-4db3-9be7-c511a1fa7549 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:59.339140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d713a5-1432-4b5d-88e1-fb656b598ae2 None admin] PATH_INFO: `/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/4f6093d0c1fc41d98c58bace81e3d8e5/roles/87751a130d004defb6bc91e40aff384d` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:59.340509 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 549/1079] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:41:59.341459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d713a5-1432-4b5d-88e1-fb656b598ae2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fc6fd42a31942688f570f1ce86be510, user_id=4f6093d0c1fc41d98c58bace81e3d8e5, role_id=87751a130d004defb6bc91e40aff384d)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:59.342996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d713a5-1432-4b5d-88e1-fb656b598ae2 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:59.347131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e5933-390e-4eff-867b-694ab8a68584 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:59.347131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e5933-390e-4eff-867b-694ab8a68584 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:59.347370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e5933-390e-4eff-867b-694ab8a68584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:59.360646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 531/1080] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:41:59 2022] PUT /identity/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/4f6093d0c1fc41d98c58bace81e3d8e5/roles/87751a130d004defb6bc91e40aff384d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:41:59.371084 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-2f9e5933-390e-4eff-867b-694ab8a68584 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:59.371084 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2f9e5933-390e-4eff-867b-694ab8a68584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:59.452151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 550/1081] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:41:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:41:59.475732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d713a5-1432-4b5d-88e1-fb656b598ae2 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:59.496264 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:59.592024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d713a5-1432-4b5d-88e1-fb656b598ae2 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:59.628344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9c756-de03-4647-9d3a-c70a75bca145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:59.629047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9c756-de03-4647-9d3a-c70a75bca145 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:59.629339 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9c756-de03-4647-9d3a-c70a75bca145 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:59.629619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9c756-de03-4647-9d3a-c70a75bca145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:59.699620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9c756-de03-4647-9d3a-c70a75bca145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:59.701075 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9c756-de03-4647-9d3a-c70a75bca145 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:59.719915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:59.811119 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 532/1082] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:41:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:41:59.812333 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e5933-390e-4eff-867b-694ab8a68584 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:59.866640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:59.916296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e5933-390e-4eff-867b-694ab8a68584 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:59.950620 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c74cbb-822c-4581-8879-4f2ae8322765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:59.951440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c74cbb-822c-4581-8879-4f2ae8322765 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:59.951726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c74cbb-822c-4581-8879-4f2ae8322765 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:59.952003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c74cbb-822c-4581-8879-4f2ae8322765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:59.962696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c74cbb-822c-4581-8879-4f2ae8322765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:59.964171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c74cbb-822c-4581-8879-4f2ae8322765 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-53c74cbb-822c-4581-8879-4f2ae8322765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:41:59.982276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:41:59.983453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:41:59.984915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:41:59.984915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:41:59.984915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:41:59.984915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:41:59.984915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 551/1083] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:41:59.989381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9c756-de03-4647-9d3a-c70a75bca145 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:00.101240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9c756-de03-4647-9d3a-c70a75bca145 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:00.118711 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:00.138130 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a99dc-9b10-4be4-bf5b-32fb9b81a3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:00.138804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a99dc-9b10-4be4-bf5b-32fb9b81a3d1 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:00.139121 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a99dc-9b10-4be4-bf5b-32fb9b81a3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:00.139402 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a99dc-9b10-4be4-bf5b-32fb9b81a3d1 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:00.140123 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a99dc-9b10-4be4-bf5b-32fb9b81a3d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:00.140434 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a99dc-9b10-4be4-bf5b-32fb9b81a3d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:00.141798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a99dc-9b10-4be4-bf5b-32fb9b81a3d1 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:00.158746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 533/1084] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:01.503154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cd34f-0187-4d70-ae0b-45b67a70d8ef None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:01.503762 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cd34f-0187-4d70-ae0b-45b67a70d8ef None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:01.503762 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cd34f-0187-4d70-ae0b-45b67a70d8ef None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:01.505140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 552/1085] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:01.511212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56fc78-4c2e-4f7f-a15f-6f3b81b80e77 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:01.511755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56fc78-4c2e-4f7f-a15f-6f3b81b80e77 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:01.511905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56fc78-4c2e-4f7f-a15f-6f3b81b80e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:01.529761 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6e56fc78-4c2e-4f7f-a15f-6f3b81b80e77 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:01.530980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6e56fc78-4c2e-4f7f-a15f-6f3b81b80e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:01.597567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 534/1086] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:01.621538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cd34f-0187-4d70-ae0b-45b67a70d8ef None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:01.700440 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c97b4-bcbf-4ac3-b13c-f5e69ce0cee1 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:01.700633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c97b4-bcbf-4ac3-b13c-f5e69ce0cee1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:01.700921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c97b4-bcbf-4ac3-b13c-f5e69ce0cee1 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:01.702299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 535/1087] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:42:01.707432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03573-bc16-42b9-8e21-ab8a6959bba4 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:01.707728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03573-bc16-42b9-8e21-ab8a6959bba4 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:01.708012 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03573-bc16-42b9-8e21-ab8a6959bba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:01.725589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cd34f-0187-4d70-ae0b-45b67a70d8ef None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:01.725760 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-c9b03573-bc16-42b9-8e21-ab8a6959bba4 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:01.726932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c9b03573-bc16-42b9-8e21-ab8a6959bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:01.728564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:01.762439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fb88e-5543-4a29-9ec9-b862ea2c0770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:01.763261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fb88e-5543-4a29-9ec9-b862ea2c0770 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:01.763548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fb88e-5543-4a29-9ec9-b862ea2c0770 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:01.763825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fb88e-5543-4a29-9ec9-b862ea2c0770 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:01.773968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fb88e-5543-4a29-9ec9-b862ea2c0770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-6b3fb88e-5543-4a29-9ec9-b862ea2c0770 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:42:01.781494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:42:01.782481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:42:01.783588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:42:01.783588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:42:01.783588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:42:01.783588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:42:01.783588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:42:01.783588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:42:01.783588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:42:01.783588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:42:01.783588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:42:01.783588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:42:01.783588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:01.783588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:42:01.783588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 553/1088] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:42:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:42:01.788693 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fb88e-5543-4a29-9ec9-b862ea2c0770 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:01.793672 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 536/1089] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:42:01.817504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b03573-bc16-42b9-8e21-ab8a6959bba4 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:01.851151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:01.887109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fb88e-5543-4a29-9ec9-b862ea2c0770 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:01.917829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b03573-bc16-42b9-8e21-ab8a6959bba4 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:01.923513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b4503-0d4a-4bd5-8185-bf784ceeb57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:01.924886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b4503-0d4a-4bd5-8185-bf784ceeb57f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:01.925170 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b4503-0d4a-4bd5-8185-bf784ceeb57f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:01.925448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b4503-0d4a-4bd5-8185-bf784ceeb57f None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:01.926206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b4503-0d4a-4bd5-8185-bf784ceeb57f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:01.926512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b4503-0d4a-4bd5-8185-bf784ceeb57f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:01.927909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b4503-0d4a-4bd5-8185-bf784ceeb57f None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:01.937483 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 554/1090] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:42:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:42:01.943673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b4503-0d4a-4bd5-8185-bf784ceeb57f None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:01.954723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c3dcc-0a66-46da-9fd0-b66640df0335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:01.955514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c3dcc-0a66-46da-9fd0-b66640df0335 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:01.955794 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c3dcc-0a66-46da-9fd0-b66640df0335 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:01.956075 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c3dcc-0a66-46da-9fd0-b66640df0335 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:01.966302 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c3dcc-0a66-46da-9fd0-b66640df0335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:42:01.969936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-bf2c3dcc-0a66-46da-9fd0-b66640df0335 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:42:01.975495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:42:01.976516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:42:01.977510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:42:01.977510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:42:01.977510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:42:01.977510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:42:01.977510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:42:01.977510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:42:01.977510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:42:01.977510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:42:01.977510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:42:01.977510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:42:01.977510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:42:01.977510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:42:01.977510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 537/1091] 10.222.0.52 () {56 vars in 1120 bytes} [Thu May 19 13:42:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:42:01.982670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c3dcc-0a66-46da-9fd0-b66640df0335 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:02.043552 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b4503-0d4a-4bd5-8185-bf784ceeb57f None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:02.076984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57e36d-f90b-40e1-9fb9-659e05242eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:02.077846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57e36d-f90b-40e1-9fb9-659e05242eb5 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:02.078129 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57e36d-f90b-40e1-9fb9-659e05242eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:02.078412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57e36d-f90b-40e1-9fb9-659e05242eb5 None admin] PATH_INFO: `/v3/users/224011d625e64851b278c7801e304edf` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:02.079363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:02.079659 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57e36d-f90b-40e1-9fb9-659e05242eb5 None admin] RBAC: Authorizing `identity:get_user(user_id=224011d625e64851b278c7801e304edf)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:02.081254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57e36d-f90b-40e1-9fb9-659e05242eb5 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:02.083227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 555/1092] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:42:01 2022] GET /identity/v3/users/224011d625e64851b278c7801e304edf => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:42:02.088446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57e36d-f90b-40e1-9fb9-659e05242eb5 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:02.089472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c3dcc-0a66-46da-9fd0-b66640df0335 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:02.124820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bec06-864d-42b8-a570-13c46bbd9348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:02.125493 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bec06-864d-42b8-a570-13c46bbd9348 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:02.125778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bec06-864d-42b8-a570-13c46bbd9348 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:02.126055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bec06-864d-42b8-a570-13c46bbd9348 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:02.126797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7bec06-864d-42b8-a570-13c46bbd9348 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:02.127134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7bec06-864d-42b8-a570-13c46bbd9348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:02.128486 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7bec06-864d-42b8-a570-13c46bbd9348 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:02.136577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 538/1093] 10.222.0.52 () {56 vars in 1115 bytes} [Thu May 19 13:42:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:42:02.141949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bec06-864d-42b8-a570-13c46bbd9348 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:02.182346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:02.189334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57e36d-f90b-40e1-9fb9-659e05242eb5 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:02.223227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2753b3-7e32-4f0c-a3c8-9950bf97201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:02.223798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2753b3-7e32-4f0c-a3c8-9950bf97201a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:02.224082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2753b3-7e32-4f0c-a3c8-9950bf97201a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:02.224355 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2753b3-7e32-4f0c-a3c8-9950bf97201a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:02.240345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bec06-864d-42b8-a570-13c46bbd9348 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:02.273990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84861adf-d989-4823-945f-5611c7162bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:02.274772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84861adf-d989-4823-945f-5611c7162bae None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:02.275087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84861adf-d989-4823-945f-5611c7162bae None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:02.275357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84861adf-d989-4823-945f-5611c7162bae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:02.286587 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84861adf-d989-4823-945f-5611c7162bae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:02.288057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84861adf-d989-4823-945f-5611c7162bae None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:02.297153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2753b3-7e32-4f0c-a3c8-9950bf97201a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:02.298616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2753b3-7e32-4f0c-a3c8-9950bf97201a None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-84861adf-d989-4823-945f-5611c7162bae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:42:02.305210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:42:02.306336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:42:02.307433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:42:02.307433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:42:02.307433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:42:02.307433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:42:02.307433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 539/1094] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:42:02.311984 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84861adf-d989-4823-945f-5611c7162bae None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:02.324988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:02.403147 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:02.411741 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 556/1095] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:42:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 13:42:02.417377 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2753b3-7e32-4f0c-a3c8-9950bf97201a None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:02.423157 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84861adf-d989-4823-945f-5611c7162bae None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:02.460366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a349d9-1912-48b7-a26b-d46bf8a303d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:02.461023 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a349d9-1912-48b7-a26b-d46bf8a303d6 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:02.461306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a349d9-1912-48b7-a26b-d46bf8a303d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:02.461578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a349d9-1912-48b7-a26b-d46bf8a303d6 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:02.462289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a349d9-1912-48b7-a26b-d46bf8a303d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:02.462591 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a349d9-1912-48b7-a26b-d46bf8a303d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:02.464061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a349d9-1912-48b7-a26b-d46bf8a303d6 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:02.476406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 540/1096] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:42:02.482084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a349d9-1912-48b7-a26b-d46bf8a303d6 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:02.525668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2753b3-7e32-4f0c-a3c8-9950bf97201a None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:02.556336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:02.559784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c89e44-438d-4a35-a506-0441dc432f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:02.560646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c89e44-438d-4a35-a506-0441dc432f02 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:02.560933 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c89e44-438d-4a35-a506-0441dc432f02 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:02.561213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c89e44-438d-4a35-a506-0441dc432f02 None admin] PATH_INFO: `/v3/projects/64fd800f6d0e4e33968655c529cfe669` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:02.562405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c89e44-438d-4a35-a506-0441dc432f02 None admin] RBAC: Authorizing `identity:get_project(project_id=64fd800f6d0e4e33968655c529cfe669)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:02.563863 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c89e44-438d-4a35-a506-0441dc432f02 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:02.566192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 557/1097] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:42:02 2022] GET /identity/v3/projects/64fd800f6d0e4e33968655c529cfe669 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:42:02.572080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c89e44-438d-4a35-a506-0441dc432f02 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:02.590149 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a349d9-1912-48b7-a26b-d46bf8a303d6 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:02.626230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2198a-cfe0-415c-a306-746a70b39899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:02.627699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2198a-cfe0-415c-a306-746a70b39899 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:02.628077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2198a-cfe0-415c-a306-746a70b39899 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:02.628353 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2198a-cfe0-415c-a306-746a70b39899 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:02.635940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2198a-cfe0-415c-a306-746a70b39899 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:02.637965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2198a-cfe0-415c-a306-746a70b39899 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:02.651008 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-a5c2198a-cfe0-415c-a306-746a70b39899 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:42:02.654799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:42:02.655923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:42:02.657354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:42:02.657354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:42:02.657354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:42:02.657354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:42:02.657354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:42:02.657354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:42:02.657354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:42:02.657354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:42:02.657354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:42:02.657354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:42:02.657354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 541/1098] 10.222.0.52 () {56 vars in 1093 bytes} [Thu May 19 13:42:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:42:02.662386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2198a-cfe0-415c-a306-746a70b39899 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:02.695989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c89e44-438d-4a35-a506-0441dc432f02 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:02.730337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d73e72-cde7-446a-b2b4-2211c389dbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:02.730928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d73e72-cde7-446a-b2b4-2211c389dbee None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:02.731216 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d73e72-cde7-446a-b2b4-2211c389dbee None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:02.731486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d73e72-cde7-446a-b2b4-2211c389dbee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:02.732307 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d73e72-cde7-446a-b2b4-2211c389dbee None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=64fd800f6d0e4e33968655c529cfe669, user.id=224011d625e64851b278c7801e304edf) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:02.732624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d73e72-cde7-446a-b2b4-2211c389dbee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:02.734047 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d73e72-cde7-446a-b2b4-2211c389dbee None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:02.752939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 558/1099] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:42:02 2022] GET /identity/v3/role_assignments?user.id=224011d625e64851b278c7801e304edf&scope.project.id=64fd800f6d0e4e33968655c529cfe669&role.id=0893bb944dd94634accd3906aa391cd2 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:02.767092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2198a-cfe0-415c-a306-746a70b39899 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:02.801951 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:02.808825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c3076-5987-4fcd-bb0e-6e0433c4c515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:02.809989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c3076-5987-4fcd-bb0e-6e0433c4c515 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:02.810492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c3076-5987-4fcd-bb0e-6e0433c4c515 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:02.810783 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c3076-5987-4fcd-bb0e-6e0433c4c515 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:02.811696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c3076-5987-4fcd-bb0e-6e0433c4c515 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:02.812018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c3076-5987-4fcd-bb0e-6e0433c4c515 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:02.813520 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c3076-5987-4fcd-bb0e-6e0433c4c515 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:02.839664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 542/1100] 10.222.0.52 () {56 vars in 1133 bytes} [Thu May 19 13:42:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:42:02.840915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d73e72-cde7-446a-b2b4-2211c389dbee None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:02.955088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d73e72-cde7-446a-b2b4-2211c389dbee None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:02.982439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:02.991827 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbc14b-76cb-4eba-b5ea-af5406774137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:02.992481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbc14b-76cb-4eba-b5ea-af5406774137 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:02.992755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbc14b-76cb-4eba-b5ea-af5406774137 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:02.993031 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbc14b-76cb-4eba-b5ea-af5406774137 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:03.068436 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbc14b-76cb-4eba-b5ea-af5406774137 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:03.069886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbc14b-76cb-4eba-b5ea-af5406774137 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:03.171838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 559/1101] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:42:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:42:03.179091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c3076-5987-4fcd-bb0e-6e0433c4c515 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:03.281925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c3076-5987-4fcd-bb0e-6e0433c4c515 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:03.319172 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050138c8-751a-43b5-9ea7-9f61c78ec61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:03.320024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050138c8-751a-43b5-9ea7-9f61c78ec61c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:03.320351 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050138c8-751a-43b5-9ea7-9f61c78ec61c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:03.320635 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050138c8-751a-43b5-9ea7-9f61c78ec61c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:03.331799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050138c8-751a-43b5-9ea7-9f61c78ec61c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:03.333262 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050138c8-751a-43b5-9ea7-9f61c78ec61c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-050138c8-751a-43b5-9ea7-9f61c78ec61c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:42:03.350625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:42:03.351636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:42:03.352534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:42:03.352534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:42:03.352534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:42:03.352534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:42:03.352534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 543/1102] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:42:03.357453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbc14b-76cb-4eba-b5ea-af5406774137 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:03.384363 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:03.457530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbc14b-76cb-4eba-b5ea-af5406774137 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:03.490284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840ec7d-b96f-47d8-9882-a621179dc6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:03.490984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840ec7d-b96f-47d8-9882-a621179dc6ee None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:03.491272 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840ec7d-b96f-47d8-9882-a621179dc6ee None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:03.491550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840ec7d-b96f-47d8-9882-a621179dc6ee None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:03.492269 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840ec7d-b96f-47d8-9882-a621179dc6ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:03.492578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840ec7d-b96f-47d8-9882-a621179dc6ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:03.493997 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840ec7d-b96f-47d8-9882-a621179dc6ee None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:03.506218 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 560/1103] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:03.511642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050138c8-751a-43b5-9ea7-9f61c78ec61c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:03.553212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:03.610445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050138c8-751a-43b5-9ea7-9f61c78ec61c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:03.645297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e82fb7-3821-4042-97d2-eeb9d9cb797c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:03.646090 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e82fb7-3821-4042-97d2-eeb9d9cb797c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:03.646374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e82fb7-3821-4042-97d2-eeb9d9cb797c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:03.646654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e82fb7-3821-4042-97d2-eeb9d9cb797c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:03.653411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e82fb7-3821-4042-97d2-eeb9d9cb797c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:03.654840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e82fb7-3821-4042-97d2-eeb9d9cb797c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-e5e82fb7-3821-4042-97d2-eeb9d9cb797c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:42:03.668623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:42:03.669684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:42:03.670612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:42:03.670612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:42:03.670612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:42:03.670612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:42:03.670612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:42:03.670612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:42:03.670612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:42:03.670612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:42:03.670612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 544/1104] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:42:03.678263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840ec7d-b96f-47d8-9882-a621179dc6ee None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:03.761795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:03.776501 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840ec7d-b96f-47d8-9882-a621179dc6ee None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:03.809542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53eafee-9569-45a1-a87b-bc3eb1cd85d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:03.810134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53eafee-9569-45a1-a87b-bc3eb1cd85d3 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:03.810420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53eafee-9569-45a1-a87b-bc3eb1cd85d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:03.810697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53eafee-9569-45a1-a87b-bc3eb1cd85d3 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:03.811632 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53eafee-9569-45a1-a87b-bc3eb1cd85d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:03.812003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53eafee-9569-45a1-a87b-bc3eb1cd85d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:03.813439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53eafee-9569-45a1-a87b-bc3eb1cd85d3 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:03.826002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 561/1105] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:42:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:03.832420 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e82fb7-3821-4042-97d2-eeb9d9cb797c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:03.929186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:03.932589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e82fb7-3821-4042-97d2-eeb9d9cb797c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:03.967810 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e81184-ecbc-48d2-8459-3848ebfe741c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:03.968469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81184-ecbc-48d2-8459-3848ebfe741c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:03.968755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81184-ecbc-48d2-8459-3848ebfe741c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:03.969031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81184-ecbc-48d2-8459-3848ebfe741c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:03.969850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e81184-ecbc-48d2-8459-3848ebfe741c None admin] RBAC: Adding query filter params (role.id=87751a130d004defb6bc91e40aff384d, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=4f6093d0c1fc41d98c58bace81e3d8e5) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:03.970162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e81184-ecbc-48d2-8459-3848ebfe741c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:03.971638 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e81184-ecbc-48d2-8459-3848ebfe741c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:03.977439 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 545/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:42:03 2022] GET /identity/v3/role_assignments?user.id=4f6093d0c1fc41d98c58bace81e3d8e5&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=87751a130d004defb6bc91e40aff384d => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:05.054927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b6acb-7c55-4714-8c13-a70d63e0e252 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:05.055456 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b6acb-7c55-4714-8c13-a70d63e0e252 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:05.055572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b6acb-7c55-4714-8c13-a70d63e0e252 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:05.056868 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 562/1107] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:05.062093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d306-cc48-40e7-974c-9cb093e04fb7 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:05.062527 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d306-cc48-40e7-974c-9cb093e04fb7 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:05.062653 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d306-cc48-40e7-974c-9cb093e04fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:05.079419 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-e5c3d306-cc48-40e7-974c-9cb093e04fb7 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:05.080626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e5c3d306-cc48-40e7-974c-9cb093e04fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:05.145247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 546/1108] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:05.170362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b6acb-7c55-4714-8c13-a70d63e0e252 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:05.269395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b6acb-7c55-4714-8c13-a70d63e0e252 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:05.302093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc9edf-7c0e-4205-b6dd-3447688feaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:05.302886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc9edf-7c0e-4205-b6dd-3447688feaab None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:05.303175 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc9edf-7c0e-4205-b6dd-3447688feaab None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:05.303452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc9edf-7c0e-4205-b6dd-3447688feaab None admin] PATH_INFO: `/v3/domains/default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:05.304620 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc9edf-7c0e-4205-b6dd-3447688feaab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:05.306087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc9edf-7c0e-4205-b6dd-3447688feaab None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:05.307670 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 563/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:42:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:05.313275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3d306-cc48-40e7-974c-9cb093e04fb7 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:05.404719 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:05.412647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3d306-cc48-40e7-974c-9cb093e04fb7 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:05.445150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a6fe1-14be-4084-a5aa-f02a777ebd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:05.445811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a6fe1-14be-4084-a5aa-f02a777ebd9a None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:05.446098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a6fe1-14be-4084-a5aa-f02a777ebd9a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:05.446378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a6fe1-14be-4084-a5aa-f02a777ebd9a None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:05.447100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a6fe1-14be-4084-a5aa-f02a777ebd9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:05.448520 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a6fe1-14be-4084-a5aa-f02a777ebd9a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:05.468309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 547/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:42:05 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:42:07.696871 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a4082-f65d-478d-974d-d6311eec4e22 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:07.697371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a4082-f65d-478d-974d-d6311eec4e22 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:07.697371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a4082-f65d-478d-974d-d6311eec4e22 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:07.698695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 564/1111] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:07.703938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791f466-4e8a-47b3-bad6-0a4958021814 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:07.704322 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791f466-4e8a-47b3-bad6-0a4958021814 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:07.704500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791f466-4e8a-47b3-bad6-0a4958021814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:07.722218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-8791f466-4e8a-47b3-bad6-0a4958021814 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:07.723427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-8791f466-4e8a-47b3-bad6-0a4958021814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:07.787946 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 548/1112] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:07.810654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a4082-f65d-478d-974d-d6311eec4e22 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:07.826626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:07.913085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a4082-f65d-478d-974d-d6311eec4e22 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:07.947422 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ecbe8-7563-437d-b59c-fbff0eac7d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:07.948202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ecbe8-7563-437d-b59c-fbff0eac7d4f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:07.948491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ecbe8-7563-437d-b59c-fbff0eac7d4f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:07.948773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ecbe8-7563-437d-b59c-fbff0eac7d4f None admin] PATH_INFO: `/v3/roles/5c2125e2125d40bca6590d50afd677d1` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:07.950245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ecbe8-7563-437d-b59c-fbff0eac7d4f None admin] RBAC: Authorizing `identity:get_role(role_id=5c2125e2125d40bca6590d50afd677d1)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:07.951640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ecbe8-7563-437d-b59c-fbff0eac7d4f None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:07.952986 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 565/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:42:07 2022] GET /identity/v3/roles/5c2125e2125d40bca6590d50afd677d1 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:07.959581 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791f466-4e8a-47b3-bad6-0a4958021814 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:08.036061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:08.055027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791f466-4e8a-47b3-bad6-0a4958021814 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:08.087788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039c7cd-08be-42bd-a7de-177600f5a296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:08.088652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039c7cd-08be-42bd-a7de-177600f5a296 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:08.088936 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039c7cd-08be-42bd-a7de-177600f5a296 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:08.089211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039c7cd-08be-42bd-a7de-177600f5a296 None admin] PATH_INFO: `/v3/users/fd8c2c57e74b42c99ba05a8de15ea12e` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:08.102338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039c7cd-08be-42bd-a7de-177600f5a296 None admin] RBAC: Authorizing `identity:get_user(user_id=fd8c2c57e74b42c99ba05a8de15ea12e)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:08.103801 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039c7cd-08be-42bd-a7de-177600f5a296 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:08.105548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 549/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:42:07 2022] GET /identity/v3/users/fd8c2c57e74b42c99ba05a8de15ea12e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:08.112284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ecbe8-7563-437d-b59c-fbff0eac7d4f None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:08.193834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:08.212306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ecbe8-7563-437d-b59c-fbff0eac7d4f None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:08.245380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1235ca-8b2d-45d7-9e80-80c0e777fd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:08.246030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1235ca-8b2d-45d7-9e80-80c0e777fd51 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:08.246304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1235ca-8b2d-45d7-9e80-80c0e777fd51 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:08.246582 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1235ca-8b2d-45d7-9e80-80c0e777fd51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:08.311487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1235ca-8b2d-45d7-9e80-80c0e777fd51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:08.312917 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1235ca-8b2d-45d7-9e80-80c0e777fd51 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:08.391037 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:08.413235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 566/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:42:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:42:08.418950 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039c7cd-08be-42bd-a7de-177600f5a296 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:08.524442 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039c7cd-08be-42bd-a7de-177600f5a296 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:08.557504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac9aa5-9e26-49ca-9c5a-66bb75dc565e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:08.558357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac9aa5-9e26-49ca-9c5a-66bb75dc565e None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:08.558642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac9aa5-9e26-49ca-9c5a-66bb75dc565e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:08.558947 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac9aa5-9e26-49ca-9c5a-66bb75dc565e None admin] PATH_INFO: `/v3/projects/64fd800f6d0e4e33968655c529cfe669` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:08.560130 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac9aa5-9e26-49ca-9c5a-66bb75dc565e None admin] RBAC: Authorizing `identity:get_project(project_id=64fd800f6d0e4e33968655c529cfe669)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:08.561550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac9aa5-9e26-49ca-9c5a-66bb75dc565e None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:08.563268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 550/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:42:08 2022] GET /identity/v3/projects/64fd800f6d0e4e33968655c529cfe669 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:08.570106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1235ca-8b2d-45d7-9e80-80c0e777fd51 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:08.666013 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1235ca-8b2d-45d7-9e80-80c0e777fd51 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:08.698622 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b0a0f-e7d2-404a-b780-60281ceef941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:08.699297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b0a0f-e7d2-404a-b780-60281ceef941 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:08.699589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b0a0f-e7d2-404a-b780-60281ceef941 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:08.699862 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b0a0f-e7d2-404a-b780-60281ceef941 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:08.700675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b0a0f-e7d2-404a-b780-60281ceef941 None admin] RBAC: Adding query filter params (role.id=5c2125e2125d40bca6590d50afd677d1, scope.project.id=64fd800f6d0e4e33968655c529cfe669, user.id=fd8c2c57e74b42c99ba05a8de15ea12e) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:08.700985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b0a0f-e7d2-404a-b780-60281ceef941 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:08.702395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b0a0f-e7d2-404a-b780-60281ceef941 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:08.708325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 567/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:42:08 2022] GET /identity/v3/role_assignments?user.id=fd8c2c57e74b42c99ba05a8de15ea12e&scope.project.id=64fd800f6d0e4e33968655c529cfe669&role.id=5c2125e2125d40bca6590d50afd677d1 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:10.883780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49196ee-5f37-49d0-aa1b-21fdb3790561 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:10.884296 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49196ee-5f37-49d0-aa1b-21fdb3790561 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:10.884392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49196ee-5f37-49d0-aa1b-21fdb3790561 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:10.885696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 551/1118] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:10.890794 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95e879-5775-40c7-91ac-df6b66daf32a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:10.891206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95e879-5775-40c7-91ac-df6b66daf32a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:10.891385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95e879-5775-40c7-91ac-df6b66daf32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:10.908617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-da95e879-5775-40c7-91ac-df6b66daf32a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:10.909807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-da95e879-5775-40c7-91ac-df6b66daf32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:10.951012 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:10.975003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 568/1119] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:10.997698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49196ee-5f37-49d0-aa1b-21fdb3790561 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:11.007055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:11.110703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49196ee-5f37-49d0-aa1b-21fdb3790561 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:11.143897 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15035726-a707-4805-85f0-8f0bde8eabea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:11.144675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15035726-a707-4805-85f0-8f0bde8eabea None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:11.144957 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15035726-a707-4805-85f0-8f0bde8eabea None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:11.145232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15035726-a707-4805-85f0-8f0bde8eabea None admin] PATH_INFO: `/v3/roles/5c2125e2125d40bca6590d50afd677d1` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:11.146688 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15035726-a707-4805-85f0-8f0bde8eabea None admin] RBAC: Authorizing `identity:get_role(role_id=5c2125e2125d40bca6590d50afd677d1)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:11.148077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15035726-a707-4805-85f0-8f0bde8eabea None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:11.149437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 552/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:42:10 2022] GET /identity/v3/roles/5c2125e2125d40bca6590d50afd677d1 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:11.155235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95e879-5775-40c7-91ac-df6b66daf32a None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:11.250852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95e879-5775-40c7-91ac-df6b66daf32a None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:11.284128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823549b-80e0-4d67-b423-64684baaeddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:11.284967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823549b-80e0-4d67-b423-64684baaeddb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:11.285244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823549b-80e0-4d67-b423-64684baaeddb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:11.285516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823549b-80e0-4d67-b423-64684baaeddb None admin] PATH_INFO: `/v3/users/fd8c2c57e74b42c99ba05a8de15ea12e` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:11.286671 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823549b-80e0-4d67-b423-64684baaeddb None admin] RBAC: Authorizing `identity:get_user(user_id=fd8c2c57e74b42c99ba05a8de15ea12e)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:11.288115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823549b-80e0-4d67-b423-64684baaeddb None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:11.289710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 569/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:42:11 2022] GET /identity/v3/users/fd8c2c57e74b42c99ba05a8de15ea12e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:11.295425 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15035726-a707-4805-85f0-8f0bde8eabea None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:11.368556 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:11.394072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15035726-a707-4805-85f0-8f0bde8eabea None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:11.427657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ebc81-27b2-4f99-b5c6-2d588473d0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:11.428489 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ebc81-27b2-4f99-b5c6-2d588473d0f8 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:11.428765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ebc81-27b2-4f99-b5c6-2d588473d0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:11.429034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ebc81-27b2-4f99-b5c6-2d588473d0f8 None admin] PATH_INFO: `/v3/projects/64fd800f6d0e4e33968655c529cfe669` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:11.430188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ebc81-27b2-4f99-b5c6-2d588473d0f8 None admin] RBAC: Authorizing `identity:get_project(project_id=64fd800f6d0e4e33968655c529cfe669)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:11.431629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ebc81-27b2-4f99-b5c6-2d588473d0f8 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:11.433293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 553/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:42:11 2022] GET /identity/v3/projects/64fd800f6d0e4e33968655c529cfe669 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:11.439141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823549b-80e0-4d67-b423-64684baaeddb None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:11.484350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:11.538909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823549b-80e0-4d67-b423-64684baaeddb None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:11.571779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750a5e9-c1b7-4e66-a0ee-41253f01ea55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:11.572643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750a5e9-c1b7-4e66-a0ee-41253f01ea55 None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:11.572920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750a5e9-c1b7-4e66-a0ee-41253f01ea55 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:11.573187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750a5e9-c1b7-4e66-a0ee-41253f01ea55 None admin] PATH_INFO: `/v3/projects/64fd800f6d0e4e33968655c529cfe669/users/fd8c2c57e74b42c99ba05a8de15ea12e/roles/5c2125e2125d40bca6590d50afd677d1` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:11.575434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750a5e9-c1b7-4e66-a0ee-41253f01ea55 None admin] RBAC: Authorizing `identity:create_grant(project_id=64fd800f6d0e4e33968655c529cfe669, user_id=fd8c2c57e74b42c99ba05a8de15ea12e, role_id=5c2125e2125d40bca6590d50afd677d1)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:11.576866 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750a5e9-c1b7-4e66-a0ee-41253f01ea55 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:11.592958 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 570/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:42:11 2022] PUT /identity/v3/projects/64fd800f6d0e4e33968655c529cfe669/users/fd8c2c57e74b42c99ba05a8de15ea12e/roles/5c2125e2125d40bca6590d50afd677d1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:42:13.778490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d190a9-10ea-46d7-9a8e-6daa7d0fecc7 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:13.779017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d190a9-10ea-46d7-9a8e-6daa7d0fecc7 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:13.779017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d190a9-10ea-46d7-9a8e-6daa7d0fecc7 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:13.780328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 554/1124] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:13.785735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393acfbe-b9a0-44d6-a85b-bcf3db111e55 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:13.786158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393acfbe-b9a0-44d6-a85b-bcf3db111e55 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:13.786296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393acfbe-b9a0-44d6-a85b-bcf3db111e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:13.803447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-393acfbe-b9a0-44d6-a85b-bcf3db111e55 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:13.804637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-393acfbe-b9a0-44d6-a85b-bcf3db111e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:13.868921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 571/1125] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:13.892841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d190a9-10ea-46d7-9a8e-6daa7d0fecc7 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:13.998254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d190a9-10ea-46d7-9a8e-6daa7d0fecc7 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:14.031317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5604cc9-95ad-4959-b1f7-bb6616b5a612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:14.032102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5604cc9-95ad-4959-b1f7-bb6616b5a612 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:14.032387 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5604cc9-95ad-4959-b1f7-bb6616b5a612 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:14.032660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5604cc9-95ad-4959-b1f7-bb6616b5a612 None admin] PATH_INFO: `/v3/roles/5c2125e2125d40bca6590d50afd677d1` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:14.034670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5604cc9-95ad-4959-b1f7-bb6616b5a612 None admin] RBAC: Authorizing `identity:get_role(role_id=5c2125e2125d40bca6590d50afd677d1)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:14.036064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5604cc9-95ad-4959-b1f7-bb6616b5a612 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:14.037421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 555/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:42:13 2022] GET /identity/v3/roles/5c2125e2125d40bca6590d50afd677d1 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:14.043281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393acfbe-b9a0-44d6-a85b-bcf3db111e55 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:14.075675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:14.141263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393acfbe-b9a0-44d6-a85b-bcf3db111e55 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:14.174704 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e5828-dec6-4645-a155-2aecac5ccf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:14.175531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5828-dec6-4645-a155-2aecac5ccf0e None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:14.175813 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5828-dec6-4645-a155-2aecac5ccf0e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:14.176089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5828-dec6-4645-a155-2aecac5ccf0e None admin] PATH_INFO: `/v3/users/fd8c2c57e74b42c99ba05a8de15ea12e` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:14.177266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e5828-dec6-4645-a155-2aecac5ccf0e None admin] RBAC: Authorizing `identity:get_user(user_id=fd8c2c57e74b42c99ba05a8de15ea12e)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:14.178684 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e5828-dec6-4645-a155-2aecac5ccf0e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:14.180337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 572/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:42:14 2022] GET /identity/v3/users/fd8c2c57e74b42c99ba05a8de15ea12e => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:14.186060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5604cc9-95ad-4959-b1f7-bb6616b5a612 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:14.198377 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:14.284748 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5604cc9-95ad-4959-b1f7-bb6616b5a612 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:14.316854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44864703-5bc6-4532-a1ab-ef6615fb9fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:14.317492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44864703-5bc6-4532-a1ab-ef6615fb9fb9 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:14.317778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44864703-5bc6-4532-a1ab-ef6615fb9fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:14.318057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44864703-5bc6-4532-a1ab-ef6615fb9fb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:14.382532 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44864703-5bc6-4532-a1ab-ef6615fb9fb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:14.383992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44864703-5bc6-4532-a1ab-ef6615fb9fb9 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:14.395570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:14.481882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 556/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:42:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:42:14.488469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e5828-dec6-4645-a155-2aecac5ccf0e None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:14.586946 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e5828-dec6-4645-a155-2aecac5ccf0e None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:14.588400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:14.621346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b10630-5697-4fdc-84e2-0c8fb471baeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:14.622445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b10630-5697-4fdc-84e2-0c8fb471baeb None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:14.622731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b10630-5697-4fdc-84e2-0c8fb471baeb None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:14.623043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b10630-5697-4fdc-84e2-0c8fb471baeb None admin] PATH_INFO: `/v3/projects/64fd800f6d0e4e33968655c529cfe669` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:14.624229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b10630-5697-4fdc-84e2-0c8fb471baeb None admin] RBAC: Authorizing `identity:get_project(project_id=64fd800f6d0e4e33968655c529cfe669)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:14.625659 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b10630-5697-4fdc-84e2-0c8fb471baeb None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:14.628319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 573/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:42:14 2022] GET /identity/v3/projects/64fd800f6d0e4e33968655c529cfe669 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:14.634404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44864703-5bc6-4532-a1ab-ef6615fb9fb9 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:14.730011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44864703-5bc6-4532-a1ab-ef6615fb9fb9 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:14.741017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:14.764700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654162d-da5a-48b3-b0a3-1595ae7ab8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:14.765339 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654162d-da5a-48b3-b0a3-1595ae7ab8f1 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:14.765622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654162d-da5a-48b3-b0a3-1595ae7ab8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:14.765899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654162d-da5a-48b3-b0a3-1595ae7ab8f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:14.766717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654162d-da5a-48b3-b0a3-1595ae7ab8f1 None admin] RBAC: Adding query filter params (role.id=5c2125e2125d40bca6590d50afd677d1, scope.project.id=64fd800f6d0e4e33968655c529cfe669, user.id=fd8c2c57e74b42c99ba05a8de15ea12e) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:14.767062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654162d-da5a-48b3-b0a3-1595ae7ab8f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:14.769141 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654162d-da5a-48b3-b0a3-1595ae7ab8f1 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:14.774759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 557/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:42:14 2022] GET /identity/v3/role_assignments?user.id=fd8c2c57e74b42c99ba05a8de15ea12e&scope.project.id=64fd800f6d0e4e33968655c529cfe669&role.id=5c2125e2125d40bca6590d50afd677d1 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:16.989150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c2702-6874-48a6-880e-a13709b452d2 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:16.989680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c2702-6874-48a6-880e-a13709b452d2 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:16.989680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c2702-6874-48a6-880e-a13709b452d2 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:16.991091 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 574/1131] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:16.996343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4903e-3d3c-40d6-9f06-9263151dc568 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:16.996758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4903e-3d3c-40d6-9f06-9263151dc568 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:16.996890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4903e-3d3c-40d6-9f06-9263151dc568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:17.014143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-42d4903e-3d3c-40d6-9f06-9263151dc568 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:17.015336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-42d4903e-3d3c-40d6-9f06-9263151dc568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:17.082645 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 558/1132] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:17.105685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c2702-6874-48a6-880e-a13709b452d2 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:17.205700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c2702-6874-48a6-880e-a13709b452d2 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:17.237723 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd674af-b55b-47e7-8ee6-d326ff2da03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:17.238494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd674af-b55b-47e7-8ee6-d326ff2da03b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:17.238773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd674af-b55b-47e7-8ee6-d326ff2da03b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:17.239073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd674af-b55b-47e7-8ee6-d326ff2da03b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:17.240284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd674af-b55b-47e7-8ee6-d326ff2da03b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:17.241647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd674af-b55b-47e7-8ee6-d326ff2da03b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:17.243264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 575/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:42:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:17.249418 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4903e-3d3c-40d6-9f06-9263151dc568 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:17.347144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4903e-3d3c-40d6-9f06-9263151dc568 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:17.349449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:17.381616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b29eb-1a9b-49df-8d67-0ca98a73a326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:17.382273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b29eb-1a9b-49df-8d67-0ca98a73a326 None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:17.382557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b29eb-1a9b-49df-8d67-0ca98a73a326 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:17.382832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b29eb-1a9b-49df-8d67-0ca98a73a326 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:17.383565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b29eb-1a9b-49df-8d67-0ca98a73a326 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:17.384987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b29eb-1a9b-49df-8d67-0ca98a73a326 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:17.413733 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 559/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:42:17 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:42:19.640355 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa0e27-1a86-4d7e-9d86-811ff9f35709 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:19.640878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa0e27-1a86-4d7e-9d86-811ff9f35709 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:19.640969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa0e27-1a86-4d7e-9d86-811ff9f35709 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:19.642272 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 576/1135] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:19.647530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ccc52-09da-421b-b6eb-64a000441d35 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:19.647965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ccc52-09da-421b-b6eb-64a000441d35 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:19.648089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ccc52-09da-421b-b6eb-64a000441d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:19.664664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-cd5ccc52-09da-421b-b6eb-64a000441d35 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:19.665783 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-cd5ccc52-09da-421b-b6eb-64a000441d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:19.730246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 560/1136] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:19.755298 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa0e27-1a86-4d7e-9d86-811ff9f35709 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:19.794753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:19.854564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa0e27-1a86-4d7e-9d86-811ff9f35709 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:19.887371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f2edc-b273-46da-92db-851df9a32afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:19.888141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f2edc-b273-46da-92db-851df9a32afc None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:19.888424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f2edc-b273-46da-92db-851df9a32afc None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:19.888699 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f2edc-b273-46da-92db-851df9a32afc None admin] PATH_INFO: `/v3/domains/default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:19.889855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f2edc-b273-46da-92db-851df9a32afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:19.891314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f2edc-b273-46da-92db-851df9a32afc None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:19.892945 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 577/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:42:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:19.898577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ccc52-09da-421b-b6eb-64a000441d35 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:19.979942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:19.994702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ccc52-09da-421b-b6eb-64a000441d35 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:20.028020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3b492-ed0b-4b4c-bdd8-09b587df2b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:20.028673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3b492-ed0b-4b4c-bdd8-09b587df2b6d None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:20.028956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3b492-ed0b-4b4c-bdd8-09b587df2b6d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:20.029233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3b492-ed0b-4b4c-bdd8-09b587df2b6d None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:20.029917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3b492-ed0b-4b4c-bdd8-09b587df2b6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:20.031358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3b492-ed0b-4b4c-bdd8-09b587df2b6d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:20.050805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 561/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:42:19 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:42:22.278896 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef78d-d3ea-4296-9a6c-b162b1e5ad92 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:22.279427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef78d-d3ea-4296-9a6c-b162b1e5ad92 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:22.279521 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef78d-d3ea-4296-9a6c-b162b1e5ad92 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:22.280812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 578/1139] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:22.286560 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698a230-e3ef-4604-8661-1f5dfc143521 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:22.286999 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698a230-e3ef-4604-8661-1f5dfc143521 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:22.287147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698a230-e3ef-4604-8661-1f5dfc143521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:22.304676 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-b698a230-e3ef-4604-8661-1f5dfc143521 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:22.305865 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b698a230-e3ef-4604-8661-1f5dfc143521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:22.371539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 562/1140] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:22.395756 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ef78d-d3ea-4296-9a6c-b162b1e5ad92 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:22.494584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ef78d-d3ea-4296-9a6c-b162b1e5ad92 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:22.502179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:22.531054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e6b4d-aad9-4959-ad02-48dc7e346995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:22.532320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e6b4d-aad9-4959-ad02-48dc7e346995 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:22.533385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e6b4d-aad9-4959-ad02-48dc7e346995 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:22.533885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e6b4d-aad9-4959-ad02-48dc7e346995 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:22.546130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e6b4d-aad9-4959-ad02-48dc7e346995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-e19e6b4d-aad9-4959-ad02-48dc7e346995 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:42:22.553847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:42:22.554998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:42:22.556065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:42:22.556065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:42:22.556065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:42:22.556065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:42:22.556065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:42:22.556065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:42:22.556065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:42:22.556065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:42:22.556065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:42:22.556065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:42:22.556065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:22.556065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:42:22.556065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 579/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:42:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:42:22.563096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698a230-e3ef-4604-8661-1f5dfc143521 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:22.631059 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:22.660692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698a230-e3ef-4604-8661-1f5dfc143521 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:22.693519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab9ca2-345d-4f04-af90-214007572b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:22.694180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab9ca2-345d-4f04-af90-214007572b0f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:22.694464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab9ca2-345d-4f04-af90-214007572b0f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:22.694742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab9ca2-345d-4f04-af90-214007572b0f None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:22.695510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab9ca2-345d-4f04-af90-214007572b0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:22.695812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab9ca2-345d-4f04-af90-214007572b0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:22.697166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab9ca2-345d-4f04-af90-214007572b0f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:22.705552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 563/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:42:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:22.711413 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e6b4d-aad9-4959-ad02-48dc7e346995 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:22.807844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e6b4d-aad9-4959-ad02-48dc7e346995 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:22.840600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7a28e-0225-41c2-9f27-d28336079988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:22.841449 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7a28e-0225-41c2-9f27-d28336079988 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:22.841729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7a28e-0225-41c2-9f27-d28336079988 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:22.842005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7a28e-0225-41c2-9f27-d28336079988 None admin] PATH_INFO: `/v3/users/84a1da50399b4c92be8df5bb0f20932f` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:22.855112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7a28e-0225-41c2-9f27-d28336079988 None admin] RBAC: Authorizing `identity:get_user(user_id=84a1da50399b4c92be8df5bb0f20932f)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:22.856547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7a28e-0225-41c2-9f27-d28336079988 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:22.858179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 580/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:42:22 2022] GET /identity/v3/users/84a1da50399b4c92be8df5bb0f20932f => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:22.864031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab9ca2-345d-4f04-af90-214007572b0f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:22.960709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab9ca2-345d-4f04-af90-214007572b0f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:22.994574 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a0281-aea4-47ec-9fe0-269a3d3b2174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:22.995237 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a0281-aea4-47ec-9fe0-269a3d3b2174 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:22.995530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a0281-aea4-47ec-9fe0-269a3d3b2174 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:22.995802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a0281-aea4-47ec-9fe0-269a3d3b2174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:22.998619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:23.061518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a0281-aea4-47ec-9fe0-269a3d3b2174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:23.062909 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a0281-aea4-47ec-9fe0-269a3d3b2174 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:23.157726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 564/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:42:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:42:23.163620 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7a28e-0225-41c2-9f27-d28336079988 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:23.188196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:23.260980 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7a28e-0225-41c2-9f27-d28336079988 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:23.293001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad085d-4fe2-4815-821f-e39d1dcb3c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:23.293841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad085d-4fe2-4815-821f-e39d1dcb3c75 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:23.294123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad085d-4fe2-4815-821f-e39d1dcb3c75 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:23.294398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad085d-4fe2-4815-821f-e39d1dcb3c75 None admin] PATH_INFO: `/v3/projects/025981a29b9c4345ab23752005a8080f` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:23.295584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad085d-4fe2-4815-821f-e39d1dcb3c75 None admin] RBAC: Authorizing `identity:get_project(project_id=025981a29b9c4345ab23752005a8080f)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:23.297007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad085d-4fe2-4815-821f-e39d1dcb3c75 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:23.299415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 581/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:42:23 2022] GET /identity/v3/projects/025981a29b9c4345ab23752005a8080f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:23.305393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a0281-aea4-47ec-9fe0-269a3d3b2174 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:23.338047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:23.403263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a0281-aea4-47ec-9fe0-269a3d3b2174 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:23.436411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f29685-324f-48e2-a492-c7d253f3cfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:23.437056 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29685-324f-48e2-a492-c7d253f3cfa3 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:23.437336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29685-324f-48e2-a492-c7d253f3cfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:23.437609 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29685-324f-48e2-a492-c7d253f3cfa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:23.438426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f29685-324f-48e2-a492-c7d253f3cfa3 None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=025981a29b9c4345ab23752005a8080f, user.id=84a1da50399b4c92be8df5bb0f20932f) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:23.438737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f29685-324f-48e2-a492-c7d253f3cfa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:23.440719 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f29685-324f-48e2-a492-c7d253f3cfa3 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:23.445962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 565/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:42:23 2022] GET /identity/v3/role_assignments?user.id=84a1da50399b4c92be8df5bb0f20932f&scope.project.id=025981a29b9c4345ab23752005a8080f&role.id=0893bb944dd94634accd3906aa391cd2 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:25.632134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901bc864-695b-4d88-9cca-ce074a58f730 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:25.632688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901bc864-695b-4d88-9cca-ce074a58f730 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:25.632688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901bc864-695b-4d88-9cca-ce074a58f730 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:25.634047 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 582/1147] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:25.639145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13617eb0-d69d-4d02-a639-c5eedb77f1ae None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:25.639563 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13617eb0-d69d-4d02-a639-c5eedb77f1ae None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:25.639703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13617eb0-d69d-4d02-a639-c5eedb77f1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:25.656170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-13617eb0-d69d-4d02-a639-c5eedb77f1ae None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:25.657279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-13617eb0-d69d-4d02-a639-c5eedb77f1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:25.721814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 566/1148] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:25.743982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901bc864-695b-4d88-9cca-ce074a58f730 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:25.833030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:25.842662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901bc864-695b-4d88-9cca-ce074a58f730 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:25.876329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488d590-08fe-4133-90ec-e2ab06d5e559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:25.877108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488d590-08fe-4133-90ec-e2ab06d5e559 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:25.877389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488d590-08fe-4133-90ec-e2ab06d5e559 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:25.877665 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488d590-08fe-4133-90ec-e2ab06d5e559 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:25.887587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488d590-08fe-4133-90ec-e2ab06d5e559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-3488d590-08fe-4133-90ec-e2ab06d5e559 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:42:25.895133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:42:25.896234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:42:25.897270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:42:25.897270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:42:25.897270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:42:25.897270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:42:25.897270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:42:25.897270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:42:25.897270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:42:25.897270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:42:25.897270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:42:25.897270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:42:25.897270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:25.897270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:42:25.897270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 583/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:42:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:42:25.902412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13617eb0-d69d-4d02-a639-c5eedb77f1ae None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:25.930066 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:25.999443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13617eb0-d69d-4d02-a639-c5eedb77f1ae None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:26.031788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb137786-72e6-44d3-8beb-16dcebf49235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:26.032442 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb137786-72e6-44d3-8beb-16dcebf49235 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:26.033264 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb137786-72e6-44d3-8beb-16dcebf49235 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:26.033603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb137786-72e6-44d3-8beb-16dcebf49235 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:26.034360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb137786-72e6-44d3-8beb-16dcebf49235 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:26.034670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb137786-72e6-44d3-8beb-16dcebf49235 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:26.036062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb137786-72e6-44d3-8beb-16dcebf49235 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:26.043849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 567/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:42:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:26.050253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488d590-08fe-4133-90ec-e2ab06d5e559 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:26.146663 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488d590-08fe-4133-90ec-e2ab06d5e559 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:26.179459 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77102f51-3da8-4c59-ab3a-c486d9f1d992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:26.180356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77102f51-3da8-4c59-ab3a-c486d9f1d992 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:26.180584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77102f51-3da8-4c59-ab3a-c486d9f1d992 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:26.180858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77102f51-3da8-4c59-ab3a-c486d9f1d992 None admin] PATH_INFO: `/v3/users/84a1da50399b4c92be8df5bb0f20932f` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:26.182036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77102f51-3da8-4c59-ab3a-c486d9f1d992 None admin] RBAC: Authorizing `identity:get_user(user_id=84a1da50399b4c92be8df5bb0f20932f)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:26.183481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77102f51-3da8-4c59-ab3a-c486d9f1d992 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:26.185087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 584/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:42:26 2022] GET /identity/v3/users/84a1da50399b4c92be8df5bb0f20932f => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:42:26.190802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb137786-72e6-44d3-8beb-16dcebf49235 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:26.281169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:26.291825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb137786-72e6-44d3-8beb-16dcebf49235 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:26.323629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9af5c-7ad5-4e85-8c12-963d68ea6388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:26.324491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9af5c-7ad5-4e85-8c12-963d68ea6388 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:26.324770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9af5c-7ad5-4e85-8c12-963d68ea6388 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:26.325045 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9af5c-7ad5-4e85-8c12-963d68ea6388 None admin] PATH_INFO: `/v3/projects/025981a29b9c4345ab23752005a8080f` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:26.326197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9af5c-7ad5-4e85-8c12-963d68ea6388 None admin] RBAC: Authorizing `identity:get_project(project_id=025981a29b9c4345ab23752005a8080f)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:26.327632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9af5c-7ad5-4e85-8c12-963d68ea6388 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:26.329291 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 568/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:42:26 2022] GET /identity/v3/projects/025981a29b9c4345ab23752005a8080f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:26.335587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77102f51-3da8-4c59-ab3a-c486d9f1d992 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:26.357856 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:26.434530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77102f51-3da8-4c59-ab3a-c486d9f1d992 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:26.466243 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1291bd-c77a-45f9-a4f7-b772d88b4380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:26.467143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1291bd-c77a-45f9-a4f7-b772d88b4380 None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:26.467421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1291bd-c77a-45f9-a4f7-b772d88b4380 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:26.467689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1291bd-c77a-45f9-a4f7-b772d88b4380 None admin] PATH_INFO: `/v3/projects/025981a29b9c4345ab23752005a8080f/users/84a1da50399b4c92be8df5bb0f20932f/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:26.469603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1291bd-c77a-45f9-a4f7-b772d88b4380 None admin] RBAC: Authorizing `identity:create_grant(project_id=025981a29b9c4345ab23752005a8080f, user_id=84a1da50399b4c92be8df5bb0f20932f, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:26.471833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1291bd-c77a-45f9-a4f7-b772d88b4380 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:26.484134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 585/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:42:26 2022] PUT /identity/v3/projects/025981a29b9c4345ab23752005a8080f/users/84a1da50399b4c92be8df5bb0f20932f/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:42:28.683350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef82ae-413b-436d-9534-27298bc72536 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:28.683864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef82ae-413b-436d-9534-27298bc72536 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:28.683864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef82ae-413b-436d-9534-27298bc72536 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:28.685192 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 569/1154] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:28.691050 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15751ef5-9867-4bc6-ba74-b4d7901f3569 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:28.691460 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15751ef5-9867-4bc6-ba74-b4d7901f3569 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:28.691619 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15751ef5-9867-4bc6-ba74-b4d7901f3569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:28.707968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-15751ef5-9867-4bc6-ba74-b4d7901f3569 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:28.709158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-15751ef5-9867-4bc6-ba74-b4d7901f3569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:28.773719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 586/1155] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:42:28.796882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef82ae-413b-436d-9534-27298bc72536 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:28.896419 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef82ae-413b-436d-9534-27298bc72536 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:28.935204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75370d-a53e-4b05-bc27-999e3eb30f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:28.935920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75370d-a53e-4b05-bc27-999e3eb30f20 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:28.936207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75370d-a53e-4b05-bc27-999e3eb30f20 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:28.936487 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75370d-a53e-4b05-bc27-999e3eb30f20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:28.946456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75370d-a53e-4b05-bc27-999e3eb30f20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-3a75370d-a53e-4b05-bc27-999e3eb30f20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:42:28.954029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:42:28.955305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:42:28.956373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:42:28.956373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:42:28.956373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:42:28.956373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:42:28.956373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:42:28.956373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:42:28.956373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:42:28.956373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:42:28.956373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:42:28.956373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:42:28.956373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:28.956373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:42:28.956373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 570/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:42:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:42:28.961762 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15751ef5-9867-4bc6-ba74-b4d7901f3569 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:28.973670 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:29.060673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15751ef5-9867-4bc6-ba74-b4d7901f3569 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:29.092805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518a317-199a-420e-82fd-a382e9de3f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:29.093457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518a317-199a-420e-82fd-a382e9de3f16 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:29.093742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518a317-199a-420e-82fd-a382e9de3f16 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:29.094019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518a317-199a-420e-82fd-a382e9de3f16 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:29.094759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518a317-199a-420e-82fd-a382e9de3f16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:29.095605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518a317-199a-420e-82fd-a382e9de3f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:29.097054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518a317-199a-420e-82fd-a382e9de3f16 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:29.104782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 587/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:42:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:29.111628 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75370d-a53e-4b05-bc27-999e3eb30f20 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:29.130648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:29.213808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75370d-a53e-4b05-bc27-999e3eb30f20 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:29.247085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16901310-78e0-49d5-a223-96c945fe1dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:29.248214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16901310-78e0-49d5-a223-96c945fe1dcf None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:29.248496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16901310-78e0-49d5-a223-96c945fe1dcf None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:29.248823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16901310-78e0-49d5-a223-96c945fe1dcf None admin] PATH_INFO: `/v3/users/84a1da50399b4c92be8df5bb0f20932f` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:29.250027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16901310-78e0-49d5-a223-96c945fe1dcf None admin] RBAC: Authorizing `identity:get_user(user_id=84a1da50399b4c92be8df5bb0f20932f)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:29.251491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16901310-78e0-49d5-a223-96c945fe1dcf None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:29.253265 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 571/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:42:29 2022] GET /identity/v3/users/84a1da50399b4c92be8df5bb0f20932f => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:29.259062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518a317-199a-420e-82fd-a382e9de3f16 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:29.327085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:29.356841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518a317-199a-420e-82fd-a382e9de3f16 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:29.388827 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593c966-fe8f-4c23-bc02-be1fbb3d1e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:29.389468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593c966-fe8f-4c23-bc02-be1fbb3d1e02 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:29.389754 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593c966-fe8f-4c23-bc02-be1fbb3d1e02 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:29.390030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593c966-fe8f-4c23-bc02-be1fbb3d1e02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:29.456852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593c966-fe8f-4c23-bc02-be1fbb3d1e02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:29.458292 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593c966-fe8f-4c23-bc02-be1fbb3d1e02 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:29.523636 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:29.553964 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 588/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:42:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:42:29.559873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16901310-78e0-49d5-a223-96c945fe1dcf None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:29.643596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:29.657993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16901310-78e0-49d5-a223-96c945fe1dcf None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:29.690342 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793b721-565b-42c0-a81d-6b414ec14abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:29.691218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b721-565b-42c0-a81d-6b414ec14abe None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:29.691499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b721-565b-42c0-a81d-6b414ec14abe None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:29.691770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b721-565b-42c0-a81d-6b414ec14abe None admin] PATH_INFO: `/v3/projects/025981a29b9c4345ab23752005a8080f` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:29.692938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793b721-565b-42c0-a81d-6b414ec14abe None admin] RBAC: Authorizing `identity:get_project(project_id=025981a29b9c4345ab23752005a8080f)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:29.694354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793b721-565b-42c0-a81d-6b414ec14abe None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:29.696067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 572/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:42:29 2022] GET /identity/v3/projects/025981a29b9c4345ab23752005a8080f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:29.702007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593c966-fe8f-4c23-bc02-be1fbb3d1e02 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:29.797098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593c966-fe8f-4c23-bc02-be1fbb3d1e02 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:29.830644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083de1e-666c-40d4-98aa-9c340f5d2f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:29.831299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083de1e-666c-40d4-98aa-9c340f5d2f32 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:29.831602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083de1e-666c-40d4-98aa-9c340f5d2f32 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:29.831873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083de1e-666c-40d4-98aa-9c340f5d2f32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:29.832683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083de1e-666c-40d4-98aa-9c340f5d2f32 None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=025981a29b9c4345ab23752005a8080f, user.id=84a1da50399b4c92be8df5bb0f20932f) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:29.832989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083de1e-666c-40d4-98aa-9c340f5d2f32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:29.834406 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083de1e-666c-40d4-98aa-9c340f5d2f32 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:29.840029 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 589/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:42:29 2022] GET /identity/v3/role_assignments?user.id=84a1da50399b4c92be8df5bb0f20932f&scope.project.id=025981a29b9c4345ab23752005a8080f&role.id=0893bb944dd94634accd3906aa391cd2 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:32.067240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c40a7-d74a-41b6-958b-0f919fd63863 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:32.067841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c40a7-d74a-41b6-958b-0f919fd63863 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:32.067841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c40a7-d74a-41b6-958b-0f919fd63863 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:32.069136 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 573/1162] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:32.074322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b91285-af7a-4f73-afac-ffe9c0561126 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:32.074745 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b91285-af7a-4f73-afac-ffe9c0561126 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:32.074914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b91285-af7a-4f73-afac-ffe9c0561126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:32.092820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-94b91285-af7a-4f73-afac-ffe9c0561126 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:32.094027 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-94b91285-af7a-4f73-afac-ffe9c0561126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:32.122870 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:32.161213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 590/1163] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:32.183383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c40a7-d74a-41b6-958b-0f919fd63863 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:32.281631 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c40a7-d74a-41b6-958b-0f919fd63863 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:32.315630 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6941943-5d1a-4a13-aec5-9f7e13c5a90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:32.316275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6941943-5d1a-4a13-aec5-9f7e13c5a90f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:32.316564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6941943-5d1a-4a13-aec5-9f7e13c5a90f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:32.316844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6941943-5d1a-4a13-aec5-9f7e13c5a90f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:32.340396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:32.383888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6941943-5d1a-4a13-aec5-9f7e13c5a90f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:32.385343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6941943-5d1a-4a13-aec5-9f7e13c5a90f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:32.483148 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 574/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:42:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:42:32.489284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b91285-af7a-4f73-afac-ffe9c0561126 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:32.585586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b91285-af7a-4f73-afac-ffe9c0561126 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:32.618024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31a79f-b427-4e3f-862b-b0548043a8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:32.618811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31a79f-b427-4e3f-862b-b0548043a8c4 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:32.619116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31a79f-b427-4e3f-862b-b0548043a8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:32.619397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31a79f-b427-4e3f-862b-b0548043a8c4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:32.626630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31a79f-b427-4e3f-862b-b0548043a8c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:32.628292 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31a79f-b427-4e3f-862b-b0548043a8c4 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-5b31a79f-b427-4e3f-862b-b0548043a8c4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:42:32.642618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:42:32.643914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:42:32.645044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 591/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Thu May 19 13:42:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:42:32.649968 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6941943-5d1a-4a13-aec5-9f7e13c5a90f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:32.702943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:32.746199 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6941943-5d1a-4a13-aec5-9f7e13c5a90f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:32.780122 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe5dcd-0b2b-4eb8-9434-d5397aa586c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:32.780767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe5dcd-0b2b-4eb8-9434-d5397aa586c2 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:32.781052 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe5dcd-0b2b-4eb8-9434-d5397aa586c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:32.781326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe5dcd-0b2b-4eb8-9434-d5397aa586c2 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:32.782039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe5dcd-0b2b-4eb8-9434-d5397aa586c2 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:32.782512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe5dcd-0b2b-4eb8-9434-d5397aa586c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:32.783920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe5dcd-0b2b-4eb8-9434-d5397aa586c2 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:32.790781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 575/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Thu May 19 13:42:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:34.996178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb65b75-1c54-49bb-92ad-d9084f27ac75 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:34.996735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb65b75-1c54-49bb-92ad-d9084f27ac75 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:34.996735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb65b75-1c54-49bb-92ad-d9084f27ac75 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:34.998334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 592/1167] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:35.004492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7b431-51be-4d79-a7f8-698b162f5a8c None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:35.004889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7b431-51be-4d79-a7f8-698b162f5a8c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:35.005063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7b431-51be-4d79-a7f8-698b162f5a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:35.022214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-c2e7b431-51be-4d79-a7f8-698b162f5a8c None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:35.023422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c2e7b431-51be-4d79-a7f8-698b162f5a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:35.087711 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 576/1168] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:35.110232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb65b75-1c54-49bb-92ad-d9084f27ac75 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:35.138202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:35.210866 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb65b75-1c54-49bb-92ad-d9084f27ac75 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:35.242905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b7ac6-8140-44e4-86bd-02714473d79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:35.243548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b7ac6-8140-44e4-86bd-02714473d79e None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:35.243829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b7ac6-8140-44e4-86bd-02714473d79e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:35.244104 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b7ac6-8140-44e4-86bd-02714473d79e None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:35.244844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b7ac6-8140-44e4-86bd-02714473d79e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:35.246237 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b7ac6-8140-44e4-86bd-02714473d79e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:35.260856 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 593/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Thu May 19 13:42:35 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:42:37.492220 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676aa34-837c-402e-b560-f4060195affe None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:37.492746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676aa34-837c-402e-b560-f4060195affe None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:37.492846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676aa34-837c-402e-b560-f4060195affe None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:37.494129 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 577/1170] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:37.499775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f5468-65f8-468c-97d9-5d84f10a3d9e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:37.500216 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f5468-65f8-468c-97d9-5d84f10a3d9e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:37.500316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f5468-65f8-468c-97d9-5d84f10a3d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:37.516557 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-886f5468-65f8-468c-97d9-5d84f10a3d9e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:37.519055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-886f5468-65f8-468c-97d9-5d84f10a3d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:37.582559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 594/1171] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:37.606735 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676aa34-837c-402e-b560-f4060195affe None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:37.656085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:37.706139 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676aa34-837c-402e-b560-f4060195affe None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:37.738824 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7f309-471a-41de-9d1e-2195bd0c8605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:37.739689 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7f309-471a-41de-9d1e-2195bd0c8605 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:37.739976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7f309-471a-41de-9d1e-2195bd0c8605 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:37.740254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7f309-471a-41de-9d1e-2195bd0c8605 None admin] PATH_INFO: `/v3/domains/acc38a3435b440d5a9273d2e8f326433` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:37.750560 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7f309-471a-41de-9d1e-2195bd0c8605 None admin] RBAC: Authorizing `identity:get_domain(domain_id=acc38a3435b440d5a9273d2e8f326433)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:37.752216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7f309-471a-41de-9d1e-2195bd0c8605 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:37.753819 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 578/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Thu May 19 13:42:37 2022] GET /identity/v3/domains/acc38a3435b440d5a9273d2e8f326433 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:37.759662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f5468-65f8-468c-97d9-5d84f10a3d9e None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:37.777202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:37.857998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f5468-65f8-468c-97d9-5d84f10a3d9e None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:37.893261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199d6ab-8ff5-495c-a69a-2c7e04c13207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:37.893923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199d6ab-8ff5-495c-a69a-2c7e04c13207 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:37.894212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199d6ab-8ff5-495c-a69a-2c7e04c13207 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:37.894489 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199d6ab-8ff5-495c-a69a-2c7e04c13207 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:37.895238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199d6ab-8ff5-495c-a69a-2c7e04c13207 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:37.896660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199d6ab-8ff5-495c-a69a-2c7e04c13207 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:37.922414 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 595/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:42:37 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:42:40.228666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bff9af-69f2-46c4-91fa-0482d4378364 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:40.229195 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bff9af-69f2-46c4-91fa-0482d4378364 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:40.229318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bff9af-69f2-46c4-91fa-0482d4378364 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:40.230589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 579/1174] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:40.236218 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e391f47-8d57-4939-8f43-9a8768884bb2 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:40.236588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e391f47-8d57-4939-8f43-9a8768884bb2 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:40.236781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e391f47-8d57-4939-8f43-9a8768884bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:40.253892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-2e391f47-8d57-4939-8f43-9a8768884bb2 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:40.255083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2e391f47-8d57-4939-8f43-9a8768884bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:40.314930 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:40.320828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 596/1175] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:40.345094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bff9af-69f2-46c4-91fa-0482d4378364 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:40.445838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bff9af-69f2-46c4-91fa-0482d4378364 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:40.455215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:40.479823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4d1fc-8fef-431c-b76d-9a6597a88422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:40.480610 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4d1fc-8fef-431c-b76d-9a6597a88422 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:40.480895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4d1fc-8fef-431c-b76d-9a6597a88422 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:40.481170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4d1fc-8fef-431c-b76d-9a6597a88422 None admin] PATH_INFO: `/v3/domains/acc38a3435b440d5a9273d2e8f326433` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:40.482342 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4d1fc-8fef-431c-b76d-9a6597a88422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=acc38a3435b440d5a9273d2e8f326433)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:40.483783 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4d1fc-8fef-431c-b76d-9a6597a88422 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:40.485961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 580/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Thu May 19 13:42:40 2022] GET /identity/v3/domains/acc38a3435b440d5a9273d2e8f326433 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:40.491686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e391f47-8d57-4939-8f43-9a8768884bb2 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:40.587138 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e391f47-8d57-4939-8f43-9a8768884bb2 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:40.628582 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ffaba-338e-4d87-bc20-7f0600cc5d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:40.629232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ffaba-338e-4d87-bc20-7f0600cc5d75 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:40.629511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ffaba-338e-4d87-bc20-7f0600cc5d75 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:40.629782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ffaba-338e-4d87-bc20-7f0600cc5d75 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:40.630479 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ffaba-338e-4d87-bc20-7f0600cc5d75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:40.631944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ffaba-338e-4d87-bc20-7f0600cc5d75 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:40.651699 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 597/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:42:40 2022] POST /identity/v3/users => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:42:42.912475 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91d1c6-81d4-4dc6-b8f9-4b824f3446da None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:42.913019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91d1c6-81d4-4dc6-b8f9-4b824f3446da None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:42.913019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91d1c6-81d4-4dc6-b8f9-4b824f3446da None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:42.914370 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 581/1178] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:42.919811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b0cc5-b3e8-4012-b777-48c71d502bb6 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:42.920243 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b0cc5-b3e8-4012-b777-48c71d502bb6 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:42.920384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b0cc5-b3e8-4012-b777-48c71d502bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:42.937480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-029b0cc5-b3e8-4012-b777-48c71d502bb6 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:42.939289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-029b0cc5-b3e8-4012-b777-48c71d502bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:42.987317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:43.006146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 598/1179] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:43.029093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91d1c6-81d4-4dc6-b8f9-4b824f3446da None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:43.129525 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91d1c6-81d4-4dc6-b8f9-4b824f3446da None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:43.163189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27ce69-5cfe-4a94-b571-fb099452af9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:43.163914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27ce69-5cfe-4a94-b571-fb099452af9f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:43.164198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27ce69-5cfe-4a94-b571-fb099452af9f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:43.164477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27ce69-5cfe-4a94-b571-fb099452af9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:43.175080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27ce69-5cfe-4a94-b571-fb099452af9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-fe27ce69-5cfe-4a94-b571-fb099452af9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:42:43.182579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:42:43.183693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:42:43.184588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:42:43.184588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:42:43.184588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:42:43.184588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:42:43.184588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:42:43.184588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:42:43.184588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:42:43.184588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:42:43.184588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:42:43.184588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:42:43.184588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:43.184588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:42:43.184588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 582/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:42:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:42:43.189802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b0cc5-b3e8-4012-b777-48c71d502bb6 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:43.300678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b0cc5-b3e8-4012-b777-48c71d502bb6 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:43.334314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eba61d-7fa8-47d8-bfae-1f88e1031e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:43.335002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eba61d-7fa8-47d8-bfae-1f88e1031e47 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:43.335288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eba61d-7fa8-47d8-bfae-1f88e1031e47 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:43.335563 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eba61d-7fa8-47d8-bfae-1f88e1031e47 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:43.336374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eba61d-7fa8-47d8-bfae-1f88e1031e47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:43.336683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eba61d-7fa8-47d8-bfae-1f88e1031e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:43.338052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eba61d-7fa8-47d8-bfae-1f88e1031e47 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:43.346035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 599/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:42:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:43.351779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27ce69-5cfe-4a94-b571-fb099452af9f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:43.387273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:43.450845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27ce69-5cfe-4a94-b571-fb099452af9f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:43.482565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e8636-6e8a-451e-bec1-16218d5f7137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:43.483448 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e8636-6e8a-451e-bec1-16218d5f7137 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:43.483737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e8636-6e8a-451e-bec1-16218d5f7137 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:43.484016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e8636-6e8a-451e-bec1-16218d5f7137 None admin] PATH_INFO: `/v3/users/6ab519697af147d49f5aafdee400a3fd` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:43.497677 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e8636-6e8a-451e-bec1-16218d5f7137 None admin] RBAC: Authorizing `identity:get_user(user_id=6ab519697af147d49f5aafdee400a3fd)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:43.499152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e8636-6e8a-451e-bec1-16218d5f7137 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:43.500811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 583/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:42:43 2022] GET /identity/v3/users/6ab519697af147d49f5aafdee400a3fd => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:43.507156 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eba61d-7fa8-47d8-bfae-1f88e1031e47 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:43.552783 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:43.604190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eba61d-7fa8-47d8-bfae-1f88e1031e47 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:43.637627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a3ba6-5476-4299-a108-0571bc80e217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:43.638274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a3ba6-5476-4299-a108-0571bc80e217 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:43.638565 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a3ba6-5476-4299-a108-0571bc80e217 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:43.638846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a3ba6-5476-4299-a108-0571bc80e217 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:43.703874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a3ba6-5476-4299-a108-0571bc80e217 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:43.705343 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a3ba6-5476-4299-a108-0571bc80e217 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:43.752985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:43.804429 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 600/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:42:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:42:43.810317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e8636-6e8a-451e-bec1-16218d5f7137 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:43.900188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:43.907758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e8636-6e8a-451e-bec1-16218d5f7137 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:43.940687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc488e6-8ffe-418f-ab00-795fcffe040d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:43.941531 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc488e6-8ffe-418f-ab00-795fcffe040d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:43.941817 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc488e6-8ffe-418f-ab00-795fcffe040d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:43.942096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc488e6-8ffe-418f-ab00-795fcffe040d None admin] PATH_INFO: `/v3/projects/642e36ddd3b04b16bcf026927fc3b171` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:43.943290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc488e6-8ffe-418f-ab00-795fcffe040d None admin] RBAC: Authorizing `identity:get_project(project_id=642e36ddd3b04b16bcf026927fc3b171)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:43.944734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc488e6-8ffe-418f-ab00-795fcffe040d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:43.946408 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 584/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:42:43 2022] GET /identity/v3/projects/642e36ddd3b04b16bcf026927fc3b171 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:43.952345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a3ba6-5476-4299-a108-0571bc80e217 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:44.047807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a3ba6-5476-4299-a108-0571bc80e217 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:44.080287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fa2c8-912e-49ce-abdc-6fdf7803727b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:44.080913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fa2c8-912e-49ce-abdc-6fdf7803727b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:44.081194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fa2c8-912e-49ce-abdc-6fdf7803727b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:44.081465 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fa2c8-912e-49ce-abdc-6fdf7803727b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:44.082273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fa2c8-912e-49ce-abdc-6fdf7803727b None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=642e36ddd3b04b16bcf026927fc3b171, user.id=6ab519697af147d49f5aafdee400a3fd) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:44.082577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fa2c8-912e-49ce-abdc-6fdf7803727b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:44.084014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fa2c8-912e-49ce-abdc-6fdf7803727b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:44.089258 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 601/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:42:43 2022] GET /identity/v3/role_assignments?user.id=6ab519697af147d49f5aafdee400a3fd&scope.project.id=642e36ddd3b04b16bcf026927fc3b171&role.id=0893bb944dd94634accd3906aa391cd2 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:46.274361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205f5f2-4d9f-468d-be9c-07737f56ed7f None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:46.274944 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205f5f2-4d9f-468d-be9c-07737f56ed7f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:46.274944 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205f5f2-4d9f-468d-be9c-07737f56ed7f None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:46.276308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 585/1186] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:46.282512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc320ae-8ab2-4fbb-abdb-0667bff3128a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:46.283021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc320ae-8ab2-4fbb-abdb-0667bff3128a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:46.283205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc320ae-8ab2-4fbb-abdb-0667bff3128a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:46.301371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-6fc320ae-8ab2-4fbb-abdb-0667bff3128a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:46.302576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-6fc320ae-8ab2-4fbb-abdb-0667bff3128a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:46.305333 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:46.367587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 602/1187] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:46.390244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205f5f2-4d9f-468d-be9c-07737f56ed7f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:46.490022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205f5f2-4d9f-468d-be9c-07737f56ed7f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:46.523175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd2b28-4117-4c2d-b591-3a633eac6616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:46.523963 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd2b28-4117-4c2d-b591-3a633eac6616 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:46.524250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd2b28-4117-4c2d-b591-3a633eac6616 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:46.524580 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd2b28-4117-4c2d-b591-3a633eac6616 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:46.534964 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd2b28-4117-4c2d-b591-3a633eac6616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-26dd2b28-4117-4c2d-b591-3a633eac6616 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:42:46.542485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:42:46.543646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:42:46.544753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:42:46.544753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:42:46.544753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:42:46.544753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:42:46.544753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:42:46.544753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:42:46.544753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:42:46.544753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:42:46.544753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:42:46.544753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:42:46.544753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:46.544753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:42:46.544753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 586/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:42:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:42:46.550526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc320ae-8ab2-4fbb-abdb-0667bff3128a None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:46.653103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc320ae-8ab2-4fbb-abdb-0667bff3128a None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:46.685934 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66323d6e-e106-4a8f-910f-ec10c9259c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:46.686617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66323d6e-e106-4a8f-910f-ec10c9259c34 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:46.686852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66323d6e-e106-4a8f-910f-ec10c9259c34 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:46.687155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66323d6e-e106-4a8f-910f-ec10c9259c34 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:46.687893 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66323d6e-e106-4a8f-910f-ec10c9259c34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:46.688197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66323d6e-e106-4a8f-910f-ec10c9259c34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:46.690526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66323d6e-e106-4a8f-910f-ec10c9259c34 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:46.698506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 603/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:42:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:46.704460 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd2b28-4117-4c2d-b591-3a633eac6616 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:46.728591 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:46.802538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd2b28-4117-4c2d-b591-3a633eac6616 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:46.834745 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c0d95-7191-478c-81e6-1b65d07c939f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:46.835619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c0d95-7191-478c-81e6-1b65d07c939f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:46.835897 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c0d95-7191-478c-81e6-1b65d07c939f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:46.836167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c0d95-7191-478c-81e6-1b65d07c939f None admin] PATH_INFO: `/v3/users/6ab519697af147d49f5aafdee400a3fd` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:46.837330 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c0d95-7191-478c-81e6-1b65d07c939f None admin] RBAC: Authorizing `identity:get_user(user_id=6ab519697af147d49f5aafdee400a3fd)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:46.838734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c0d95-7191-478c-81e6-1b65d07c939f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:46.840362 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 587/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:42:46 2022] GET /identity/v3/users/6ab519697af147d49f5aafdee400a3fd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:46.846671 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66323d6e-e106-4a8f-910f-ec10c9259c34 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:46.849523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:46.945433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66323d6e-e106-4a8f-910f-ec10c9259c34 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:46.977554 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b167c5c-1a51-40a5-a000-bdceb4ae0187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:46.978394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b167c5c-1a51-40a5-a000-bdceb4ae0187 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:46.978677 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b167c5c-1a51-40a5-a000-bdceb4ae0187 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:46.978989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b167c5c-1a51-40a5-a000-bdceb4ae0187 None admin] PATH_INFO: `/v3/projects/642e36ddd3b04b16bcf026927fc3b171` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:46.980146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b167c5c-1a51-40a5-a000-bdceb4ae0187 None admin] RBAC: Authorizing `identity:get_project(project_id=642e36ddd3b04b16bcf026927fc3b171)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:46.981567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b167c5c-1a51-40a5-a000-bdceb4ae0187 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:46.983293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 604/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:42:46 2022] GET /identity/v3/projects/642e36ddd3b04b16bcf026927fc3b171 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:46.989549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c0d95-7191-478c-81e6-1b65d07c939f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:47.077214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:47.087794 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c0d95-7191-478c-81e6-1b65d07c939f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:47.119709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df7d99-554f-4e86-9e56-024af1720149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:47.120578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df7d99-554f-4e86-9e56-024af1720149 None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:47.120859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df7d99-554f-4e86-9e56-024af1720149 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:47.121130 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df7d99-554f-4e86-9e56-024af1720149 None admin] PATH_INFO: `/v3/projects/642e36ddd3b04b16bcf026927fc3b171/users/6ab519697af147d49f5aafdee400a3fd/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:47.123358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df7d99-554f-4e86-9e56-024af1720149 None admin] RBAC: Authorizing `identity:create_grant(project_id=642e36ddd3b04b16bcf026927fc3b171, user_id=6ab519697af147d49f5aafdee400a3fd, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:47.124767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df7d99-554f-4e86-9e56-024af1720149 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:47.137466 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 588/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:42:46 2022] PUT /identity/v3/projects/642e36ddd3b04b16bcf026927fc3b171/users/6ab519697af147d49f5aafdee400a3fd/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:42:49.409589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1c3bd-711b-4c79-8b4c-93a0ac701a30 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:49.410052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1c3bd-711b-4c79-8b4c-93a0ac701a30 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:49.410147 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1c3bd-711b-4c79-8b4c-93a0ac701a30 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:49.411541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 605/1193] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:42:49.416814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc3f45-ad6f-475f-b810-05588f36e5b1 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:49.417209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc3f45-ad6f-475f-b810-05588f36e5b1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:49.417372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc3f45-ad6f-475f-b810-05588f36e5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:49.434434 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-49cc3f45-ad6f-475f-b810-05588f36e5b1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:49.435644 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-49cc3f45-ad6f-475f-b810-05588f36e5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:49.500322 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 589/1194] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:42:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:42:49.523662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1c3bd-711b-4c79-8b4c-93a0ac701a30 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:49.590831 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:49.621721 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1c3bd-711b-4c79-8b4c-93a0ac701a30 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:49.654341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510acfa6-cdf2-45a8-9d08-0835f7d8e731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:49.655148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510acfa6-cdf2-45a8-9d08-0835f7d8e731 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:49.655432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510acfa6-cdf2-45a8-9d08-0835f7d8e731 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:49.655704 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510acfa6-cdf2-45a8-9d08-0835f7d8e731 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:49.665640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510acfa6-cdf2-45a8-9d08-0835f7d8e731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-510acfa6-cdf2-45a8-9d08-0835f7d8e731 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:42:49.673528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:42:49.674666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:42:49.675760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:42:49.675760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:42:49.675760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:42:49.675760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:42:49.675760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:42:49.675760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:42:49.675760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:42:49.675760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:42:49.675760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:42:49.675760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:42:49.675760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:42:49.675760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:42:49.675760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 606/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:42:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:42:49.680648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc3f45-ad6f-475f-b810-05588f36e5b1 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:49.758128 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:49.777549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc3f45-ad6f-475f-b810-05588f36e5b1 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:49.810548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee25a3-ac58-4798-ad7e-6881b6bc86b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:49.811231 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee25a3-ac58-4798-ad7e-6881b6bc86b7 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:49.811514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee25a3-ac58-4798-ad7e-6881b6bc86b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:49.811790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee25a3-ac58-4798-ad7e-6881b6bc86b7 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:49.812529 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee25a3-ac58-4798-ad7e-6881b6bc86b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:49.812835 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee25a3-ac58-4798-ad7e-6881b6bc86b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:49.814893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee25a3-ac58-4798-ad7e-6881b6bc86b7 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:49.822679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 590/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:42:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:49.828458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510acfa6-cdf2-45a8-9d08-0835f7d8e731 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:49.926358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510acfa6-cdf2-45a8-9d08-0835f7d8e731 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:49.953414 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:49.959401 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50644b-cce6-4de0-ac13-cb431b187525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:49.960251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50644b-cce6-4de0-ac13-cb431b187525 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:49.960529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50644b-cce6-4de0-ac13-cb431b187525 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:49.960797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50644b-cce6-4de0-ac13-cb431b187525 None admin] PATH_INFO: `/v3/users/6ab519697af147d49f5aafdee400a3fd` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:49.961963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50644b-cce6-4de0-ac13-cb431b187525 None admin] RBAC: Authorizing `identity:get_user(user_id=6ab519697af147d49f5aafdee400a3fd)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:49.963429 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50644b-cce6-4de0-ac13-cb431b187525 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:49.965019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 607/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Thu May 19 13:42:49 2022] GET /identity/v3/users/6ab519697af147d49f5aafdee400a3fd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:49.971371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee25a3-ac58-4798-ad7e-6881b6bc86b7 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:50.067985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee25a3-ac58-4798-ad7e-6881b6bc86b7 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:50.099920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30861b59-d7e4-4b42-8457-c36e5a252ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:50.100547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30861b59-d7e4-4b42-8457-c36e5a252ee6 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:50.100829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30861b59-d7e4-4b42-8457-c36e5a252ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:50.101104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30861b59-d7e4-4b42-8457-c36e5a252ee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:50.112673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:50.167241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30861b59-d7e4-4b42-8457-c36e5a252ee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:50.168663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30861b59-d7e4-4b42-8457-c36e5a252ee6 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:50.262587 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 591/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:42:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:42:50.268915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50644b-cce6-4de0-ac13-cb431b187525 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:50.364292 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50644b-cce6-4de0-ac13-cb431b187525 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:50.398573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54220b73-551a-42b0-95ab-f49b88e40dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:50.399458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54220b73-551a-42b0-95ab-f49b88e40dc3 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:50.399743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54220b73-551a-42b0-95ab-f49b88e40dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:50.400019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54220b73-551a-42b0-95ab-f49b88e40dc3 None admin] PATH_INFO: `/v3/projects/642e36ddd3b04b16bcf026927fc3b171` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:50.401187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54220b73-551a-42b0-95ab-f49b88e40dc3 None admin] RBAC: Authorizing `identity:get_project(project_id=642e36ddd3b04b16bcf026927fc3b171)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:50.402597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54220b73-551a-42b0-95ab-f49b88e40dc3 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:50.404303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 608/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Thu May 19 13:42:50 2022] GET /identity/v3/projects/642e36ddd3b04b16bcf026927fc3b171 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:42:50.411066 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30861b59-d7e4-4b42-8457-c36e5a252ee6 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:50.451612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:50.506975 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30861b59-d7e4-4b42-8457-c36e5a252ee6 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:50.540451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a4142-d8e5-4804-a4a4-bc3aea40f9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:50.541029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a4142-d8e5-4804-a4a4-bc3aea40f9b5 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:50.541310 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a4142-d8e5-4804-a4a4-bc3aea40f9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:50.541580 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a4142-d8e5-4804-a4a4-bc3aea40f9b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:50.542391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a4142-d8e5-4804-a4a4-bc3aea40f9b5 None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=642e36ddd3b04b16bcf026927fc3b171, user.id=6ab519697af147d49f5aafdee400a3fd) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:42:50.542699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a4142-d8e5-4804-a4a4-bc3aea40f9b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:50.544133 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a4142-d8e5-4804-a4a4-bc3aea40f9b5 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:50.550202 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 592/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:42:50 2022] GET /identity/v3/role_assignments?user.id=6ab519697af147d49f5aafdee400a3fd&scope.project.id=642e36ddd3b04b16bcf026927fc3b171&role.id=0893bb944dd94634accd3906aa391cd2 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:43:00.396722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=67 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:00.547365 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=47 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:50.925572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eac5f5-0d0c-4324-af8e-cf29f6f5795c None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:50.926206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eac5f5-0d0c-4324-af8e-cf29f6f5795c None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:50.926206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eac5f5-0d0c-4324-af8e-cf29f6f5795c None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:50.927726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 609/1201] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:43:50.933420 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4df0-071a-47be-b4cf-5c302aaa925a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:50.933924 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4df0-071a-47be-b4cf-5c302aaa925a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:50.934023 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4df0-071a-47be-b4cf-5c302aaa925a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:50.955552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-57ba4df0-071a-47be-b4cf-5c302aaa925a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:43:50.956766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-57ba4df0-071a-47be-b4cf-5c302aaa925a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:43:51.039858 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 593/1202] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:43:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:43:51.064240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eac5f5-0d0c-4324-af8e-cf29f6f5795c None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:51.209227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eac5f5-0d0c-4324-af8e-cf29f6f5795c None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:51.248201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73854925-58cb-422b-8dec-b7633db414d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:51.249008 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73854925-58cb-422b-8dec-b7633db414d3 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:51.249424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73854925-58cb-422b-8dec-b7633db414d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:51.249638 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73854925-58cb-422b-8dec-b7633db414d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:51.262204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73854925-58cb-422b-8dec-b7633db414d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:51.263857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73854925-58cb-422b-8dec-b7633db414d3 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-73854925-58cb-422b-8dec-b7633db414d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:43:51.287713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:43:51.288835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:43:51.294920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:43:51.294920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:43:51.294920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:43:51.294920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:43:51.294920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 610/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:43:51.299731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba4df0-071a-47be-b4cf-5c302aaa925a None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:51.457245 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba4df0-071a-47be-b4cf-5c302aaa925a None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:51.494506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:51.502426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aac21c-3e55-4663-a37f-3b5b8aa7ec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:51.503137 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac21c-3e55-4663-a37f-3b5b8aa7ec06 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:51.503514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac21c-3e55-4663-a37f-3b5b8aa7ec06 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:51.503800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac21c-3e55-4663-a37f-3b5b8aa7ec06 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:51.504546 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aac21c-3e55-4663-a37f-3b5b8aa7ec06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:43:51.504862 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aac21c-3e55-4663-a37f-3b5b8aa7ec06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:51.506350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aac21c-3e55-4663-a37f-3b5b8aa7ec06 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:51.521244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 594/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:43:51.527438 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73854925-58cb-422b-8dec-b7633db414d3 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:51.587495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:51.649143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73854925-58cb-422b-8dec-b7633db414d3 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:51.694397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa26426-585a-4555-b10c-92d6c8de0cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:51.695087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa26426-585a-4555-b10c-92d6c8de0cc5 None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:51.695377 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa26426-585a-4555-b10c-92d6c8de0cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:51.695657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa26426-585a-4555-b10c-92d6c8de0cc5 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:51.696373 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa26426-585a-4555-b10c-92d6c8de0cc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:51.698005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa26426-585a-4555-b10c-92d6c8de0cc5 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:51.728632 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 611/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Thu May 19 13:43:51 2022] POST /identity/v3/users => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:43:54.145757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902c892-e20f-429e-8c19-56057581fa13 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:54.146352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902c892-e20f-429e-8c19-56057581fa13 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:54.146352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902c892-e20f-429e-8c19-56057581fa13 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:54.148104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 595/1206] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:43:54.153802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20ab5b-fca6-4c6c-baf5-ff2cceb1fba5 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:54.154324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20ab5b-fca6-4c6c-baf5-ff2cceb1fba5 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:54.154422 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20ab5b-fca6-4c6c-baf5-ff2cceb1fba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:54.173968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-3e20ab5b-fca6-4c6c-baf5-ff2cceb1fba5 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:43:54.175293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3e20ab5b-fca6-4c6c-baf5-ff2cceb1fba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:43:54.262547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 612/1207] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:43:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:43:54.292807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902c892-e20f-429e-8c19-56057581fa13 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:54.410230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902c892-e20f-429e-8c19-56057581fa13 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:54.450780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec291573-e02e-4163-bd74-36e0278ba122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:54.451686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec291573-e02e-4163-bd74-36e0278ba122 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:54.451943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec291573-e02e-4163-bd74-36e0278ba122 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:54.452228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec291573-e02e-4163-bd74-36e0278ba122 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:54.465647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec291573-e02e-4163-bd74-36e0278ba122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-ec291573-e02e-4163-bd74-36e0278ba122 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:43:54.474572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:43:54.475746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:43:54.476802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:43:54.476802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:43:54.476802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:43:54.476802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:43:54.476802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:43:54.476802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:43:54.476802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:43:54.476802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:43:54.476802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:43:54.476802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:43:54.476802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:43:54.476802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:43:54.476802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 596/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:43:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:43:54.482103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20ab5b-fca6-4c6c-baf5-ff2cceb1fba5 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:54.527075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:54.602225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20ab5b-fca6-4c6c-baf5-ff2cceb1fba5 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:54.646600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae1c84-9b22-492d-8dc6-7fbb3fe4121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:54.647299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae1c84-9b22-492d-8dc6-7fbb3fe4121a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:54.647584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae1c84-9b22-492d-8dc6-7fbb3fe4121a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:54.647860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae1c84-9b22-492d-8dc6-7fbb3fe4121a None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:54.648662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae1c84-9b22-492d-8dc6-7fbb3fe4121a None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:43:54.648981 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae1c84-9b22-492d-8dc6-7fbb3fe4121a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:54.650423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae1c84-9b22-492d-8dc6-7fbb3fe4121a None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:54.660364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 613/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:43:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:43:54.666281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec291573-e02e-4163-bd74-36e0278ba122 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:54.725568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:54.784820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec291573-e02e-4163-bd74-36e0278ba122 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:54.829043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716b151-7ea5-49f9-ac16-38633e5c3a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:54.830006 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716b151-7ea5-49f9-ac16-38633e5c3a9d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:54.830298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716b151-7ea5-49f9-ac16-38633e5c3a9d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:54.830569 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716b151-7ea5-49f9-ac16-38633e5c3a9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:54.841858 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716b151-7ea5-49f9-ac16-38633e5c3a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:54.843368 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716b151-7ea5-49f9-ac16-38633e5c3a9d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-7716b151-7ea5-49f9-ac16-38633e5c3a9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:43:54.861855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:43:54.863227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:43:54.864307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:43:54.864307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:43:54.864307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:43:54.864307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:43:54.864307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 597/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:43:54.870774 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae1c84-9b22-492d-8dc6-7fbb3fe4121a None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:54.987707 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:54.990143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae1c84-9b22-492d-8dc6-7fbb3fe4121a None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:55.036095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d91c02-4341-4fa2-ad76-467095c858a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:55.036916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d91c02-4341-4fa2-ad76-467095c858a7 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:55.037199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d91c02-4341-4fa2-ad76-467095c858a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:55.037488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d91c02-4341-4fa2-ad76-467095c858a7 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:55.038293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d91c02-4341-4fa2-ad76-467095c858a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:43:55.038573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d91c02-4341-4fa2-ad76-467095c858a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:55.039998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d91c02-4341-4fa2-ad76-467095c858a7 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:55.054594 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 614/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:43:55.060256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716b151-7ea5-49f9-ac16-38633e5c3a9d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:55.185522 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716b151-7ea5-49f9-ac16-38633e5c3a9d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:55.187351 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:55.226456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafd046-40b4-4ab1-81be-7a8f2e5734f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:55.227285 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafd046-40b4-4ab1-81be-7a8f2e5734f7 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:55.227558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafd046-40b4-4ab1-81be-7a8f2e5734f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:55.227833 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafd046-40b4-4ab1-81be-7a8f2e5734f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:55.238680 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafd046-40b4-4ab1-81be-7a8f2e5734f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:55.240360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafd046-40b4-4ab1-81be-7a8f2e5734f7 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-4aafd046-40b4-4ab1-81be-7a8f2e5734f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:43:55.259284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:43:55.260793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:43:55.261881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:43:55.261881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:43:55.261881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:43:55.261881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:43:55.261881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:43:55.261881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:43:55.261881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:43:55.261881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:43:55.261881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 13:43:55.261881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:43:55.261881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 598/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:43:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:43:55.267661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d91c02-4341-4fa2-ad76-467095c858a7 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:55.386280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d91c02-4341-4fa2-ad76-467095c858a7 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:55.425253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a3b22-ba09-44e7-8aea-af70f929104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:55.425984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a3b22-ba09-44e7-8aea-af70f929104c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:55.426216 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a3b22-ba09-44e7-8aea-af70f929104c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:55.426520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a3b22-ba09-44e7-8aea-af70f929104c None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:55.427353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a3b22-ba09-44e7-8aea-af70f929104c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:43:55.427678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a3b22-ba09-44e7-8aea-af70f929104c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:55.429087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a3b22-ba09-44e7-8aea-af70f929104c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:55.447870 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 615/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Thu May 19 13:43:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:43:55.455037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafd046-40b4-4ab1-81be-7a8f2e5734f7 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:55.592136 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafd046-40b4-4ab1-81be-7a8f2e5734f7 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:55.634726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e95ac-32f9-48b5-819c-037c48e52266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:55.635456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e95ac-32f9-48b5-819c-037c48e52266 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:55.635721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e95ac-32f9-48b5-819c-037c48e52266 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:55.635998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e95ac-32f9-48b5-819c-037c48e52266 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:55.680606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:55.733864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e95ac-32f9-48b5-819c-037c48e52266 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:55.735490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e95ac-32f9-48b5-819c-037c48e52266 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:55.874246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 599/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:43:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 13:43:55.881341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a3b22-ba09-44e7-8aea-af70f929104c None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:55.902652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:56.001604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a3b22-ba09-44e7-8aea-af70f929104c None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:56.041478 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d900ad-f770-4883-aa59-d056066f54ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:56.042277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d900ad-f770-4883-aa59-d056066f54ab None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:56.042561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d900ad-f770-4883-aa59-d056066f54ab None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:56.042836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d900ad-f770-4883-aa59-d056066f54ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:56.053893 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d900ad-f770-4883-aa59-d056066f54ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:56.055442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d900ad-f770-4883-aa59-d056066f54ab None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-61d900ad-f770-4883-aa59-d056066f54ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:43:56.074753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:43:56.082151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:43:56.083085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:43:56.083085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:43:56.083085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:43:56.083085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:43:56.083085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 616/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:43:56.083361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e95ac-32f9-48b5-819c-037c48e52266 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:56.152588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:56.194410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e95ac-32f9-48b5-819c-037c48e52266 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:56.229742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7c253-ddef-4aa4-8354-d396e7a6231d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:56.230405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7c253-ddef-4aa4-8354-d396e7a6231d None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:56.230686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7c253-ddef-4aa4-8354-d396e7a6231d None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:56.230988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7c253-ddef-4aa4-8354-d396e7a6231d None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:56.231705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7c253-ddef-4aa4-8354-d396e7a6231d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:43:56.232008 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7c253-ddef-4aa4-8354-d396e7a6231d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:56.233396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7c253-ddef-4aa4-8354-d396e7a6231d None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:56.244877 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 600/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:43:56.251217 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d900ad-f770-4883-aa59-d056066f54ab None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:56.341181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:56.377063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d900ad-f770-4883-aa59-d056066f54ab None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:56.416259 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578eb92f-6f94-48cd-bd4f-77f07a693f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:56.416966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578eb92f-6f94-48cd-bd4f-77f07a693f93 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:56.417250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578eb92f-6f94-48cd-bd4f-77f07a693f93 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:56.417524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578eb92f-6f94-48cd-bd4f-77f07a693f93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:56.425742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578eb92f-6f94-48cd-bd4f-77f07a693f93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:56.427251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578eb92f-6f94-48cd-bd4f-77f07a693f93 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-578eb92f-6f94-48cd-bd4f-77f07a693f93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:43:56.444011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:43:56.445076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:43:56.446568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:43:56.446568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:43:56.446568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:43:56.446568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:43:56.446568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:43:56.446568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:43:56.446568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:43:56.446568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:43:56.446568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 617/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:43:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:43:56.452065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7c253-ddef-4aa4-8354-d396e7a6231d None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:56.568906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7c253-ddef-4aa4-8354-d396e7a6231d None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:56.595262 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:56.609345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd59217-aca3-41f1-88f6-31719cc4106b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:56.610166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd59217-aca3-41f1-88f6-31719cc4106b None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:56.610550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd59217-aca3-41f1-88f6-31719cc4106b None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:56.610987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd59217-aca3-41f1-88f6-31719cc4106b None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:56.611859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd59217-aca3-41f1-88f6-31719cc4106b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:43:56.612246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd59217-aca3-41f1-88f6-31719cc4106b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:56.613741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd59217-aca3-41f1-88f6-31719cc4106b None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:56.632513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 601/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:43:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:43:56.634260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578eb92f-6f94-48cd-bd4f-77f07a693f93 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:56.745383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578eb92f-6f94-48cd-bd4f-77f07a693f93 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:56.781002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:56.788542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d6b81-bd53-42c0-93c1-ec5012f7ae2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:56.789200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d6b81-bd53-42c0-93c1-ec5012f7ae2c None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:56.789486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d6b81-bd53-42c0-93c1-ec5012f7ae2c None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:56.789763 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d6b81-bd53-42c0-93c1-ec5012f7ae2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:56.790587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d6b81-bd53-42c0-93c1-ec5012f7ae2c None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=b105f8163ce84a9ba725e6b04d348446) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:43:56.790932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d6b81-bd53-42c0-93c1-ec5012f7ae2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:56.792620 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d6b81-bd53-42c0-93c1-ec5012f7ae2c None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:56.800071 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 618/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:43:56 2022] GET /identity/v3/role_assignments?user.id=b105f8163ce84a9ba725e6b04d348446&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:43:59.130907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf84b59-6538-4ed6-9149-95bb4cb566b5 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:59.130907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf84b59-6538-4ed6-9149-95bb4cb566b5 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:59.130907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf84b59-6538-4ed6-9149-95bb4cb566b5 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:59.133369 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 602/1220] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:43:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:43:59.139928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d03b9-c238-4955-a555-b2cf3f4b6c20 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:59.140412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d03b9-c238-4955-a555-b2cf3f4b6c20 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:59.140412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d03b9-c238-4955-a555-b2cf3f4b6c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:59.160129 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-9b9d03b9-c238-4955-a555-b2cf3f4b6c20 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:43:59.161364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9b9d03b9-c238-4955-a555-b2cf3f4b6c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:43:59.242492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 619/1221] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:43:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:43:59.268164 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf84b59-6538-4ed6-9149-95bb4cb566b5 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:59.384491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf84b59-6538-4ed6-9149-95bb4cb566b5 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:59.421068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28afdc-a05c-492a-a842-d02bae033c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:59.421869 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28afdc-a05c-492a-a842-d02bae033c78 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:59.422185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28afdc-a05c-492a-a842-d02bae033c78 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:59.422461 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28afdc-a05c-492a-a842-d02bae033c78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:59.433854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28afdc-a05c-492a-a842-d02bae033c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-0d28afdc-a05c-492a-a842-d02bae033c78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:43:59.442182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:43:59.443270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:43:59.444758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:43:59.444758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:43:59.444758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:43:59.444758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:43:59.444758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:43:59.444758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:43:59.444758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:43:59.444758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:43:59.444758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:43:59.444758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:43:59.444758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:43:59.444758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:43:59.444758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 603/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:43:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:43:59.453263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d03b9-c238-4955-a555-b2cf3f4b6c20 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:59.572668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d03b9-c238-4955-a555-b2cf3f4b6c20 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:59.600215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:59.612143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17ab35-61fd-42d6-b995-77773f2da55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:59.612824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17ab35-61fd-42d6-b995-77773f2da55d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:59.613255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17ab35-61fd-42d6-b995-77773f2da55d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:59.613538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17ab35-61fd-42d6-b995-77773f2da55d None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:59.614290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17ab35-61fd-42d6-b995-77773f2da55d None admin] RBAC: Adding query filter params (name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:43:59.614630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17ab35-61fd-42d6-b995-77773f2da55d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:59.616142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17ab35-61fd-42d6-b995-77773f2da55d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:59.626599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 620/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:43:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:43:59.632996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28afdc-a05c-492a-a842-d02bae033c78 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:59.686802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:59.754054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28afdc-a05c-492a-a842-d02bae033c78 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:59.791516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ce659-dc29-4315-965f-13c844d270a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:59.792297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ce659-dc29-4315-965f-13c844d270a3 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:59.792571 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ce659-dc29-4315-965f-13c844d270a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:59.792842 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ce659-dc29-4315-965f-13c844d270a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:59.804269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ce659-dc29-4315-965f-13c844d270a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:59.805795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ce659-dc29-4315-965f-13c844d270a3 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-9e4ce659-dc29-4315-965f-13c844d270a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:43:59.823847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:43:59.825017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:43:59.826123 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:43:59.826123 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:43:59.826123 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:43:59.826123 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:43:59.826123 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 604/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:43:59.831135 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17ab35-61fd-42d6-b995-77773f2da55d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:59.938019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17ab35-61fd-42d6-b995-77773f2da55d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:59.975697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb7340-ef31-40da-b38b-2c0a60707e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:59.976634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb7340-ef31-40da-b38b-2c0a60707e5d None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:59.976934 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb7340-ef31-40da-b38b-2c0a60707e5d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:59.977294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb7340-ef31-40da-b38b-2c0a60707e5d None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:59.978089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb7340-ef31-40da-b38b-2c0a60707e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:43:59.978398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb7340-ef31-40da-b38b-2c0a60707e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:59.979901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb7340-ef31-40da-b38b-2c0a60707e5d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:59.998423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 621/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:43:59.999726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ce659-dc29-4315-965f-13c844d270a3 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:00.106411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:00.114058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ce659-dc29-4315-965f-13c844d270a3 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:00.154732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcd138-0615-441c-a16f-1758fa0f5a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:00.155635 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcd138-0615-441c-a16f-1758fa0f5a2c None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:00.155872 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcd138-0615-441c-a16f-1758fa0f5a2c None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:00.156152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcd138-0615-441c-a16f-1758fa0f5a2c None admin] PATH_INFO: `/v3/users/placement` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:00.165781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcd138-0615-441c-a16f-1758fa0f5a2c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:00.167359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcd138-0615-441c-a16f-1758fa0f5a2c None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-2ffcd138-0615-441c-a16f-1758fa0f5a2c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:44:00.182749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:00.184053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:00.185389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:00.185389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:00.185389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:44:00.185389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:44:00.185389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:44:00.185389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:44:00.185389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:44:00.185389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:44:00.185389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 13:44:00.185389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:00.185389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 605/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:43:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:00.189879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb7340-ef31-40da-b38b-2c0a60707e5d None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:00.243441 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:00.304790 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb7340-ef31-40da-b38b-2c0a60707e5d None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:00.344065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b836d-2f77-4782-ade0-2a890188cb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:00.345228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b836d-2f77-4782-ade0-2a890188cb5b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:00.345592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b836d-2f77-4782-ade0-2a890188cb5b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:00.346001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b836d-2f77-4782-ade0-2a890188cb5b None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:00.347162 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b836d-2f77-4782-ade0-2a890188cb5b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:00.347681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b836d-2f77-4782-ade0-2a890188cb5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:00.349290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b836d-2f77-4782-ade0-2a890188cb5b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:00.369380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 622/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Thu May 19 13:44:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:00.376175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcd138-0615-441c-a16f-1758fa0f5a2c None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:00.493331 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcd138-0615-441c-a16f-1758fa0f5a2c None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:00.532726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b192a99-b11c-4f7e-85b8-2b14ec302998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:00.533677 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b192a99-b11c-4f7e-85b8-2b14ec302998 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:00.534092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b192a99-b11c-4f7e-85b8-2b14ec302998 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:00.534514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b192a99-b11c-4f7e-85b8-2b14ec302998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:00.546702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b192a99-b11c-4f7e-85b8-2b14ec302998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:00.548327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b192a99-b11c-4f7e-85b8-2b14ec302998 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-5b192a99-b11c-4f7e-85b8-2b14ec302998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:44:00.567490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:00.568650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:00.569902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:44:00.569902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:44:00.569902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:00.569902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:00.569902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 606/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:00.574713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b836d-2f77-4782-ade0-2a890188cb5b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:00.698844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b836d-2f77-4782-ade0-2a890188cb5b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:00.701439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:00.735880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259edbe-ca99-468b-b68e-f4f4820eaed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:00.736595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259edbe-ca99-468b-b68e-f4f4820eaed1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:00.736878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259edbe-ca99-468b-b68e-f4f4820eaed1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:00.737154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259edbe-ca99-468b-b68e-f4f4820eaed1 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:00.737939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259edbe-ca99-468b-b68e-f4f4820eaed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:00.738188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259edbe-ca99-468b-b68e-f4f4820eaed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:00.739593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259edbe-ca99-468b-b68e-f4f4820eaed1 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:00.753338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 623/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:00.758905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b192a99-b11c-4f7e-85b8-2b14ec302998 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:00.768764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:00.867309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b192a99-b11c-4f7e-85b8-2b14ec302998 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:00.905808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1946b7d-afe9-4217-b255-b6226902a1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:00.907469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946b7d-afe9-4217-b255-b6226902a1cf None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:00.907770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946b7d-afe9-4217-b255-b6226902a1cf None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:00.908073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946b7d-afe9-4217-b255-b6226902a1cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:00.916139 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1946b7d-afe9-4217-b255-b6226902a1cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:00.918183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1946b7d-afe9-4217-b255-b6226902a1cf None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-f1946b7d-afe9-4217-b255-b6226902a1cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:44:00.933588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:00.934696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:00.935830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:44:00.935830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:44:00.935830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:44:00.935830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:44:00.935830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:44:00.935830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:44:00.935830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:44:00.935830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:00.935830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 607/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:44:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:00.942421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259edbe-ca99-468b-b68e-f4f4820eaed1 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:01.051998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259edbe-ca99-468b-b68e-f4f4820eaed1 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:01.090793 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa2ac5-de36-40cd-9e87-dfc49383d376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:01.091540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2ac5-de36-40cd-9e87-dfc49383d376 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:01.091866 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2ac5-de36-40cd-9e87-dfc49383d376 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:01.092226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2ac5-de36-40cd-9e87-dfc49383d376 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:01.093019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa2ac5-de36-40cd-9e87-dfc49383d376 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:01.093331 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa2ac5-de36-40cd-9e87-dfc49383d376 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:01.094784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa2ac5-de36-40cd-9e87-dfc49383d376 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:01.107497 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 624/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:44:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:01.113703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1946b7d-afe9-4217-b255-b6226902a1cf None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:01.198415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:01.239078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1946b7d-afe9-4217-b255-b6226902a1cf None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:01.277725 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5303c-de0a-4389-bd2d-3639a24a8eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:01.278942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5303c-de0a-4389-bd2d-3639a24a8eea None admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:01.279517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5303c-de0a-4389-bd2d-3639a24a8eea None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:01.279982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5303c-de0a-4389-bd2d-3639a24a8eea None admin] PATH_INFO: `/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/b105f8163ce84a9ba725e6b04d348446/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:01.298119 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5303c-de0a-4389-bd2d-3639a24a8eea None admin] RBAC: Authorizing `identity:create_grant(project_id=2fc6fd42a31942688f570f1ce86be510, user_id=b105f8163ce84a9ba725e6b04d348446, role_id=c0cbd2d231594ea2bf7bb013fbf9ad1f)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:01.299771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5303c-de0a-4389-bd2d-3639a24a8eea None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:01.316090 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 608/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:44:01 2022] PUT /identity/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/b105f8163ce84a9ba725e6b04d348446/roles/c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:44:03.850903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c6e50-382c-41c6-b181-35b8b96a8edf None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:03.851455 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c6e50-382c-41c6-b181-35b8b96a8edf None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:03.851455 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c6e50-382c-41c6-b181-35b8b96a8edf None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:03.852763 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 625/1233] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:44:03.858384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b74ed-9377-448f-a3dc-6451a4b5b9d1 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:03.858887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b74ed-9377-448f-a3dc-6451a4b5b9d1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:03.858991 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b74ed-9377-448f-a3dc-6451a4b5b9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:03.878477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-c18b74ed-9377-448f-a3dc-6451a4b5b9d1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:44:03.879921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c18b74ed-9377-448f-a3dc-6451a4b5b9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:44:03.954668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 609/1234] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:44:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:44:03.979499 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c6e50-382c-41c6-b181-35b8b96a8edf None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:04.008859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:04.096376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c6e50-382c-41c6-b181-35b8b96a8edf None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:04.135581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13be88-2290-428a-a5b8-408269d21107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:04.136568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13be88-2290-428a-a5b8-408269d21107 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:04.136929 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13be88-2290-428a-a5b8-408269d21107 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:04.137313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13be88-2290-428a-a5b8-408269d21107 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:04.149574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13be88-2290-428a-a5b8-408269d21107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-2a13be88-2290-428a-a5b8-408269d21107 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:44:04.157728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:04.159063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:04.160228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:04.160228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:04.160228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:04.160228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:04.160228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:44:04.160228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:44:04.160228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:44:04.160228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:44:04.160228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:44:04.160228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:44:04.160228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:44:04.160228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:44:04.160228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 626/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Thu May 19 13:44:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:04.165643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b74ed-9377-448f-a3dc-6451a4b5b9d1 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:04.222411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:04.280971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b74ed-9377-448f-a3dc-6451a4b5b9d1 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:04.317143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012329f-394a-4a6c-97d2-7e82499e0964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:04.317863 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012329f-394a-4a6c-97d2-7e82499e0964 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:04.318182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012329f-394a-4a6c-97d2-7e82499e0964 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:04.318463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012329f-394a-4a6c-97d2-7e82499e0964 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:04.319239 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012329f-394a-4a6c-97d2-7e82499e0964 None admin] RBAC: Adding query filter params (name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:04.319549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012329f-394a-4a6c-97d2-7e82499e0964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:04.320963 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012329f-394a-4a6c-97d2-7e82499e0964 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:04.330852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 610/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Thu May 19 13:44:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:04.337480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13be88-2290-428a-a5b8-408269d21107 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:04.431306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:04.449001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13be88-2290-428a-a5b8-408269d21107 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:04.492126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae33eeb-1ee7-43c9-9da9-546be4f886c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:04.493250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae33eeb-1ee7-43c9-9da9-546be4f886c8 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:04.493765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae33eeb-1ee7-43c9-9da9-546be4f886c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:04.494149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae33eeb-1ee7-43c9-9da9-546be4f886c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:04.508500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae33eeb-1ee7-43c9-9da9-546be4f886c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:04.509962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae33eeb-1ee7-43c9-9da9-546be4f886c8 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-5ae33eeb-1ee7-43c9-9da9-546be4f886c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:44:04.529937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:04.531306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:04.532969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:44:04.532969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:44:04.532969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:04.532969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:44:04.532969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 627/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:44:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:04.537829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012329f-394a-4a6c-97d2-7e82499e0964 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:04.654642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012329f-394a-4a6c-97d2-7e82499e0964 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:04.665014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:04.695966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b10ee1-0151-407a-917a-5673ed5522e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:04.696629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b10ee1-0151-407a-917a-5673ed5522e5 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:04.696931 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b10ee1-0151-407a-917a-5673ed5522e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:04.697189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b10ee1-0151-407a-917a-5673ed5522e5 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:04.697909 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b10ee1-0151-407a-917a-5673ed5522e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:04.698212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b10ee1-0151-407a-917a-5673ed5522e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:04.699598 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b10ee1-0151-407a-917a-5673ed5522e5 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:04.714163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 611/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:04.720522 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae33eeb-1ee7-43c9-9da9-546be4f886c8 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:04.830534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae33eeb-1ee7-43c9-9da9-546be4f886c8 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:04.869331 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d675a-ce6a-4338-933f-5c52a5cdbba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:04.870170 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d675a-ce6a-4338-933f-5c52a5cdbba3 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:04.870558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d675a-ce6a-4338-933f-5c52a5cdbba3 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:04.870835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d675a-ce6a-4338-933f-5c52a5cdbba3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:04.874564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:04.881901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d675a-ce6a-4338-933f-5c52a5cdbba3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:04.883902 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d675a-ce6a-4338-933f-5c52a5cdbba3 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-b66d675a-ce6a-4338-933f-5c52a5cdbba3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:44:04.903371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:04.904820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:04.906178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:04.906178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:04.906178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:44:04.906178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:44:04.906178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:44:04.906178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:44:04.906178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:44:04.906178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:44:04.906178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 13:44:04.906178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:44:04.906178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 628/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:44:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:04.910992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b10ee1-0151-407a-917a-5673ed5522e5 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:05.034788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b10ee1-0151-407a-917a-5673ed5522e5 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:05.076875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905b3c1-c99b-40d2-b13b-08e0c3b95624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:05.077637 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905b3c1-c99b-40d2-b13b-08e0c3b95624 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:05.077980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905b3c1-c99b-40d2-b13b-08e0c3b95624 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:05.078321 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905b3c1-c99b-40d2-b13b-08e0c3b95624 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:05.080203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905b3c1-c99b-40d2-b13b-08e0c3b95624 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:05.080520 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905b3c1-c99b-40d2-b13b-08e0c3b95624 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:05.082385 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905b3c1-c99b-40d2-b13b-08e0c3b95624 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:05.111603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 612/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Thu May 19 13:44:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:05.112668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d675a-ce6a-4338-933f-5c52a5cdbba3 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:05.248884 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d675a-ce6a-4338-933f-5c52a5cdbba3 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:05.300181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e205e7-289f-4d80-b19c-c27aabea8c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:05.301285 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e205e7-289f-4d80-b19c-c27aabea8c7f None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:05.301452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e205e7-289f-4d80-b19c-c27aabea8c7f None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:05.301778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e205e7-289f-4d80-b19c-c27aabea8c7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:05.392112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:05.401595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e205e7-289f-4d80-b19c-c27aabea8c7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:05.403496 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e205e7-289f-4d80-b19c-c27aabea8c7f None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:05.522114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 629/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:44:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:44:05.523505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905b3c1-c99b-40d2-b13b-08e0c3b95624 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:05.553799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:05.641160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905b3c1-c99b-40d2-b13b-08e0c3b95624 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:05.691072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87600be7-95f2-4706-9fae-f7ad6709e559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:05.692018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87600be7-95f2-4706-9fae-f7ad6709e559 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:05.692397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87600be7-95f2-4706-9fae-f7ad6709e559 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:05.692734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87600be7-95f2-4706-9fae-f7ad6709e559 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:05.705250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87600be7-95f2-4706-9fae-f7ad6709e559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:05.706755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87600be7-95f2-4706-9fae-f7ad6709e559 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-87600be7-95f2-4706-9fae-f7ad6709e559 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:44:05.730175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:05.731393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:05.736110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:44:05.736110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:44:05.736110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:05.736110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:05.736110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 613/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:44:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:44:05.738389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e205e7-289f-4d80-b19c-c27aabea8c7f None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:05.838724 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:05.845661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e205e7-289f-4d80-b19c-c27aabea8c7f None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:05.883366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d8ee6-0aed-4b42-8e2d-28e9c1355759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:05.884059 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d8ee6-0aed-4b42-8e2d-28e9c1355759 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:05.884313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d8ee6-0aed-4b42-8e2d-28e9c1355759 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:05.884559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d8ee6-0aed-4b42-8e2d-28e9c1355759 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:05.885268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d8ee6-0aed-4b42-8e2d-28e9c1355759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:05.885572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d8ee6-0aed-4b42-8e2d-28e9c1355759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:05.886958 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d8ee6-0aed-4b42-8e2d-28e9c1355759 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:05.898417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 630/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:05.904048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87600be7-95f2-4706-9fae-f7ad6709e559 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:05.995867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:06.022178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87600be7-95f2-4706-9fae-f7ad6709e559 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:06.060209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b76264-2d7d-46a1-bcb0-3ff6a30b2edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:06.061126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b76264-2d7d-46a1-bcb0-3ff6a30b2edc None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:06.061456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b76264-2d7d-46a1-bcb0-3ff6a30b2edc None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:06.061735 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b76264-2d7d-46a1-bcb0-3ff6a30b2edc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:06.069914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b76264-2d7d-46a1-bcb0-3ff6a30b2edc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:06.071576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b76264-2d7d-46a1-bcb0-3ff6a30b2edc None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-06b76264-2d7d-46a1-bcb0-3ff6a30b2edc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:44:06.086463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:06.087731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:06.088715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:44:06.088715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:44:06.088715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:44:06.088715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:44:06.088715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:44:06.088715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:44:06.088715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:44:06.088715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:06.088715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 614/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:44:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:06.093639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d8ee6-0aed-4b42-8e2d-28e9c1355759 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:06.223966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d8ee6-0aed-4b42-8e2d-28e9c1355759 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:06.258018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3cb68-33bb-430e-8c1e-2f2777369fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:06.258686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3cb68-33bb-430e-8c1e-2f2777369fd1 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:06.259007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3cb68-33bb-430e-8c1e-2f2777369fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:06.259286 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3cb68-33bb-430e-8c1e-2f2777369fd1 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:06.260249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3cb68-33bb-430e-8c1e-2f2777369fd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:06.260570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3cb68-33bb-430e-8c1e-2f2777369fd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:06.261984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3cb68-33bb-430e-8c1e-2f2777369fd1 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:06.274812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 631/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:44:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:06.281951 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b76264-2d7d-46a1-bcb0-3ff6a30b2edc None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:06.390242 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b76264-2d7d-46a1-bcb0-3ff6a30b2edc None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:06.426341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefbd83-89f1-4fc1-ae11-26f22a91577f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:06.427028 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefbd83-89f1-4fc1-ae11-26f22a91577f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:06.427313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefbd83-89f1-4fc1-ae11-26f22a91577f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:06.427611 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefbd83-89f1-4fc1-ae11-26f22a91577f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:06.428415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefbd83-89f1-4fc1-ae11-26f22a91577f None admin] RBAC: Adding query filter params (role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=b105f8163ce84a9ba725e6b04d348446) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:06.428726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefbd83-89f1-4fc1-ae11-26f22a91577f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:06.430211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefbd83-89f1-4fc1-ae11-26f22a91577f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:06.436274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 615/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:44:06 2022] GET /identity/v3/role_assignments?user.id=b105f8163ce84a9ba725e6b04d348446&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=c0cbd2d231594ea2bf7bb013fbf9ad1f => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:08.830325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380da09-d000-43c8-b74f-d002a73f3e05 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:08.830849 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380da09-d000-43c8-b74f-d002a73f3e05 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:08.830849 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380da09-d000-43c8-b74f-d002a73f3e05 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:08.832214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 632/1247] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:44:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:44:08.837622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724027f8-b102-43f9-b70d-c67166f4d4e0 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:08.838104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724027f8-b102-43f9-b70d-c67166f4d4e0 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:08.838237 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724027f8-b102-43f9-b70d-c67166f4d4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:08.843237 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:08.856842 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-724027f8-b102-43f9-b70d-c67166f4d4e0 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:44:08.858038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-724027f8-b102-43f9-b70d-c67166f4d4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:44:08.930584 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 616/1248] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:44:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:44:08.955869 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380da09-d000-43c8-b74f-d002a73f3e05 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:08.975675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:09.071007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380da09-d000-43c8-b74f-d002a73f3e05 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:09.111625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10276dd4-5e00-47c7-9d93-c6bbc59b311a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:09.112323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10276dd4-5e00-47c7-9d93-c6bbc59b311a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:09.112589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10276dd4-5e00-47c7-9d93-c6bbc59b311a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:09.112858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10276dd4-5e00-47c7-9d93-c6bbc59b311a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:09.124325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10276dd4-5e00-47c7-9d93-c6bbc59b311a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-10276dd4-5e00-47c7-9d93-c6bbc59b311a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:44:09.132049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:09.133581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:09.134680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:09.134680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:09.134680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:09.134680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:09.134680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:44:09.134680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:44:09.134680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:44:09.134680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:44:09.134680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:44:09.134680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:44:09.134680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:44:09.134680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:44:09.134680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 633/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:44:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:44:09.139537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724027f8-b102-43f9-b70d-c67166f4d4e0 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:09.251346 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724027f8-b102-43f9-b70d-c67166f4d4e0 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:09.282247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:09.287799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e747bc-1b70-42e7-9755-a1ff11a1e677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:09.288723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e747bc-1b70-42e7-9755-a1ff11a1e677 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:09.289078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e747bc-1b70-42e7-9755-a1ff11a1e677 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:09.289433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e747bc-1b70-42e7-9755-a1ff11a1e677 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:09.290251 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e747bc-1b70-42e7-9755-a1ff11a1e677 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:09.290559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e747bc-1b70-42e7-9755-a1ff11a1e677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:09.292069 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e747bc-1b70-42e7-9755-a1ff11a1e677 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:09.300850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 617/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:44:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:44:09.306530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10276dd4-5e00-47c7-9d93-c6bbc59b311a None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:09.402676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:09.429920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10276dd4-5e00-47c7-9d93-c6bbc59b311a None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:09.476199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada33fd2-7c52-418f-8e1d-e8c1ba752f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:09.477429 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada33fd2-7c52-418f-8e1d-e8c1ba752f74 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:09.477857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada33fd2-7c52-418f-8e1d-e8c1ba752f74 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:09.478213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada33fd2-7c52-418f-8e1d-e8c1ba752f74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:09.493736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada33fd2-7c52-418f-8e1d-e8c1ba752f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:09.495798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada33fd2-7c52-418f-8e1d-e8c1ba752f74 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-ada33fd2-7c52-418f-8e1d-e8c1ba752f74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:44:09.519507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:09.520566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:09.521703 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:44:09.521703 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:44:09.521703 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:09.521703 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:44:09.521703 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 634/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:44:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:09.526588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e747bc-1b70-42e7-9755-a1ff11a1e677 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:09.637186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e747bc-1b70-42e7-9755-a1ff11a1e677 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:09.678255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80226944-1036-423d-9733-d4d95f6d5dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:09.678849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80226944-1036-423d-9733-d4d95f6d5dbb None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:09.679163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80226944-1036-423d-9733-d4d95f6d5dbb None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:09.679437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80226944-1036-423d-9733-d4d95f6d5dbb None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:09.680176 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80226944-1036-423d-9733-d4d95f6d5dbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:09.680530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80226944-1036-423d-9733-d4d95f6d5dbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:09.681957 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80226944-1036-423d-9733-d4d95f6d5dbb None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:09.695017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 618/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:09.700627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada33fd2-7c52-418f-8e1d-e8c1ba752f74 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:09.810961 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada33fd2-7c52-418f-8e1d-e8c1ba752f74 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:09.841278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:09.850548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f0138-f80d-4ef8-a000-a49a5d106625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:09.851547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f0138-f80d-4ef8-a000-a49a5d106625 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:09.851971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f0138-f80d-4ef8-a000-a49a5d106625 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:09.852306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f0138-f80d-4ef8-a000-a49a5d106625 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:09.861668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f0138-f80d-4ef8-a000-a49a5d106625 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:09.863144 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f0138-f80d-4ef8-a000-a49a5d106625 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-8d5f0138-f80d-4ef8-a000-a49a5d106625 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:44:09.881380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:09.882467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:09.883717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:09.883717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:09.883717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:44:09.883717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:44:09.883717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:44:09.883717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:44:09.883717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:44:09.883717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:44:09.883717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 13:44:09.883717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:44:09.883717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 635/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:44:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:09.889712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80226944-1036-423d-9733-d4d95f6d5dbb None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:09.949331 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:09.996360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80226944-1036-423d-9733-d4d95f6d5dbb None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:10.032012 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedebce-349b-4c1f-aa5e-a19d79f1ee88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:10.032912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedebce-349b-4c1f-aa5e-a19d79f1ee88 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:10.033208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedebce-349b-4c1f-aa5e-a19d79f1ee88 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:10.033479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedebce-349b-4c1f-aa5e-a19d79f1ee88 None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:10.034324 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dedebce-349b-4c1f-aa5e-a19d79f1ee88 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:10.034637 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dedebce-349b-4c1f-aa5e-a19d79f1ee88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:10.036093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dedebce-349b-4c1f-aa5e-a19d79f1ee88 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:10.054447 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 619/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Thu May 19 13:44:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:10.060661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f0138-f80d-4ef8-a000-a49a5d106625 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:10.174446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f0138-f80d-4ef8-a000-a49a5d106625 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:10.209962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d37dac-5733-42f9-a435-ca2cd592a75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:10.210651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d37dac-5733-42f9-a435-ca2cd592a75b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:10.211039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d37dac-5733-42f9-a435-ca2cd592a75b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:10.211389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d37dac-5733-42f9-a435-ca2cd592a75b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:10.270081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:10.285100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d37dac-5733-42f9-a435-ca2cd592a75b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:10.286582 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d37dac-5733-42f9-a435-ca2cd592a75b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:10.395912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 636/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:44:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:44:10.401863 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedebce-349b-4c1f-aa5e-a19d79f1ee88 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:10.511437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedebce-349b-4c1f-aa5e-a19d79f1ee88 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:10.526542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:10.549219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff012df-c48a-4c5a-96e1-833b674179d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:10.550024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff012df-c48a-4c5a-96e1-833b674179d2 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:10.550307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff012df-c48a-4c5a-96e1-833b674179d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:10.550583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff012df-c48a-4c5a-96e1-833b674179d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:10.561612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff012df-c48a-4c5a-96e1-833b674179d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:10.563143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff012df-c48a-4c5a-96e1-833b674179d2 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-aff012df-c48a-4c5a-96e1-833b674179d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:44:10.581449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:10.582506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:10.583450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:44:10.583450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:44:10.583450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:10.583450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:10.583450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 620/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:44:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:10.588533 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d37dac-5733-42f9-a435-ca2cd592a75b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:10.680574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:10.694360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d37dac-5733-42f9-a435-ca2cd592a75b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:10.730972 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7185a8-d240-44db-98e6-eb9c6a6134b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:10.731649 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7185a8-d240-44db-98e6-eb9c6a6134b0 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:10.731935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7185a8-d240-44db-98e6-eb9c6a6134b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:10.732214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7185a8-d240-44db-98e6-eb9c6a6134b0 None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:10.732950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7185a8-d240-44db-98e6-eb9c6a6134b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:10.733258 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7185a8-d240-44db-98e6-eb9c6a6134b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:10.734696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7185a8-d240-44db-98e6-eb9c6a6134b0 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:10.747069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 637/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:10.753165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff012df-c48a-4c5a-96e1-833b674179d2 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:10.857920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff012df-c48a-4c5a-96e1-833b674179d2 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:10.895907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2bee2-5504-46ef-b2c3-6810ec279d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:10.896706 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2bee2-5504-46ef-b2c3-6810ec279d72 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:10.896987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2bee2-5504-46ef-b2c3-6810ec279d72 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:10.897258 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2bee2-5504-46ef-b2c3-6810ec279d72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:10.903961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2bee2-5504-46ef-b2c3-6810ec279d72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:10.905379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2bee2-5504-46ef-b2c3-6810ec279d72 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-4de2bee2-5504-46ef-b2c3-6810ec279d72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:44:10.918939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:10.919974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:10.920887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:44:10.920887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:44:10.920887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:44:10.920887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:44:10.920887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:44:10.920887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:44:10.920887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:44:10.920887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:10.920887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 621/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:44:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:10.927500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7185a8-d240-44db-98e6-eb9c6a6134b0 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:11.031869 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7185a8-d240-44db-98e6-eb9c6a6134b0 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:11.064775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf3776-3b9b-4ca2-b143-423e498194e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:11.065417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf3776-3b9b-4ca2-b143-423e498194e4 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:11.065696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf3776-3b9b-4ca2-b143-423e498194e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:11.065968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf3776-3b9b-4ca2-b143-423e498194e4 None admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:11.066740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf3776-3b9b-4ca2-b143-423e498194e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:11.067079 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf3776-3b9b-4ca2-b143-423e498194e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:11.068485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf3776-3b9b-4ca2-b143-423e498194e4 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:11.080605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 638/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:44:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:11.086753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2bee2-5504-46ef-b2c3-6810ec279d72 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:11.113965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:11.198793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2bee2-5504-46ef-b2c3-6810ec279d72 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:11.231166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d66061-94be-4328-a869-8defae86d440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:11.231809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d66061-94be-4328-a869-8defae86d440 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:11.232093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d66061-94be-4328-a869-8defae86d440 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:11.232368 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d66061-94be-4328-a869-8defae86d440 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:11.233186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d66061-94be-4328-a869-8defae86d440 None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=b105f8163ce84a9ba725e6b04d348446) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:11.233494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d66061-94be-4328-a869-8defae86d440 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:11.235629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d66061-94be-4328-a869-8defae86d440 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:11.240954 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 622/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:44:11 2022] GET /identity/v3/role_assignments?user.id=b105f8163ce84a9ba725e6b04d348446&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=0893bb944dd94634accd3906aa391cd2 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:13.395976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f11b07-68e4-4e53-9d13-2f63b7f076df None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:13.396497 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f11b07-68e4-4e53-9d13-2f63b7f076df None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:13.396609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f11b07-68e4-4e53-9d13-2f63b7f076df None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:13.397883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 639/1261] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:44:13.403101 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd814a-2a17-4bc3-a671-fd331b6dd511 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:13.403468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd814a-2a17-4bc3-a671-fd331b6dd511 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:13.403659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd814a-2a17-4bc3-a671-fd331b6dd511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:13.420269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-53bd814a-2a17-4bc3-a671-fd331b6dd511 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:44:13.421444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-53bd814a-2a17-4bc3-a671-fd331b6dd511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:44:13.486981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 623/1262] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:44:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:44:13.509894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f11b07-68e4-4e53-9d13-2f63b7f076df None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:13.520947 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:13.608769 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f11b07-68e4-4e53-9d13-2f63b7f076df None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:13.641977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a5612-84a8-4c95-82a8-0c6799c49301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:13.642748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a5612-84a8-4c95-82a8-0c6799c49301 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:13.643064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a5612-84a8-4c95-82a8-0c6799c49301 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:13.643340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a5612-84a8-4c95-82a8-0c6799c49301 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:13.653279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a5612-84a8-4c95-82a8-0c6799c49301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-b35a5612-84a8-4c95-82a8-0c6799c49301 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:44:13.661359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:13.662447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:13.663420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:13.663420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:13.663420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:13.663420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:13.663420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:44:13.663420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:44:13.663420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:44:13.663420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:44:13.663420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:44:13.663420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:44:13.663420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:44:13.663420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:44:13.663420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 640/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:44:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:13.668410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd814a-2a17-4bc3-a671-fd331b6dd511 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:13.686629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:13.764616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd814a-2a17-4bc3-a671-fd331b6dd511 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:13.797810 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4302f6-5e50-4014-b8ff-99178c5bb083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:13.798458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4302f6-5e50-4014-b8ff-99178c5bb083 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:13.798741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4302f6-5e50-4014-b8ff-99178c5bb083 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:13.799043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4302f6-5e50-4014-b8ff-99178c5bb083 None admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:13.799776 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4302f6-5e50-4014-b8ff-99178c5bb083 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:13.800078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4302f6-5e50-4014-b8ff-99178c5bb083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:13.801422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4302f6-5e50-4014-b8ff-99178c5bb083 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:13.809153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 624/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:44:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:13.815352 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a5612-84a8-4c95-82a8-0c6799c49301 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:13.878873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:13.911641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a5612-84a8-4c95-82a8-0c6799c49301 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:13.943754 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c800f-e8f3-45e9-b234-a0f70dcb1540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:13.944522 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c800f-e8f3-45e9-b234-a0f70dcb1540 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:13.944800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c800f-e8f3-45e9-b234-a0f70dcb1540 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:13.945070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c800f-e8f3-45e9-b234-a0f70dcb1540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:13.955709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c800f-e8f3-45e9-b234-a0f70dcb1540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:13.957137 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c800f-e8f3-45e9-b234-a0f70dcb1540 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-643c800f-e8f3-45e9-b234-a0f70dcb1540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:44:13.973605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:13.974662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:13.975788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:44:13.975788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:44:13.975788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:13.975788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:44:13.975788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 641/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:44:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:13.980394 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4302f6-5e50-4014-b8ff-99178c5bb083 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:14.058496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:14.077995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4302f6-5e50-4014-b8ff-99178c5bb083 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:14.110552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3f1c0-654b-4ddd-a9bc-b2e79c5fd9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:14.111227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3f1c0-654b-4ddd-a9bc-b2e79c5fd9e9 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:14.111517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3f1c0-654b-4ddd-a9bc-b2e79c5fd9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:14.111795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3f1c0-654b-4ddd-a9bc-b2e79c5fd9e9 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:14.112507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3f1c0-654b-4ddd-a9bc-b2e79c5fd9e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:14.112814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3f1c0-654b-4ddd-a9bc-b2e79c5fd9e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:14.114162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3f1c0-654b-4ddd-a9bc-b2e79c5fd9e9 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:14.128633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 625/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:14.134279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c800f-e8f3-45e9-b234-a0f70dcb1540 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:14.244590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c800f-e8f3-45e9-b234-a0f70dcb1540 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:14.264036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:14.278150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63158565-1081-4311-8406-d2d13a2ee65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:14.278967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63158565-1081-4311-8406-d2d13a2ee65e None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:14.279255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63158565-1081-4311-8406-d2d13a2ee65e None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:14.279530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63158565-1081-4311-8406-d2d13a2ee65e None admin] PATH_INFO: `/v3/users/placement` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:14.286726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63158565-1081-4311-8406-d2d13a2ee65e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:14.288201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63158565-1081-4311-8406-d2d13a2ee65e None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-63158565-1081-4311-8406-d2d13a2ee65e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:44:14.302624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:14.303761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:14.304859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:14.304859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:14.304859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:44:14.304859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:44:14.304859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:44:14.304859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:44:14.304859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:44:14.304859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:44:14.304859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 13:44:14.304859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:44:14.304859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 642/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:44:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:14.309546 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3f1c0-654b-4ddd-a9bc-b2e79c5fd9e9 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:14.406317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3f1c0-654b-4ddd-a9bc-b2e79c5fd9e9 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:14.439890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833c991-8cd4-4089-932e-7e6531d8fc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:14.440547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833c991-8cd4-4089-932e-7e6531d8fc1a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:14.440827 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833c991-8cd4-4089-932e-7e6531d8fc1a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:14.441098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833c991-8cd4-4089-932e-7e6531d8fc1a None admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:14.441919 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833c991-8cd4-4089-932e-7e6531d8fc1a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:14.442225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833c991-8cd4-4089-932e-7e6531d8fc1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:14.443664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833c991-8cd4-4089-932e-7e6531d8fc1a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:14.451905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:14.461343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 626/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Thu May 19 13:44:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:14.466892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63158565-1081-4311-8406-d2d13a2ee65e None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:14.564112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63158565-1081-4311-8406-d2d13a2ee65e None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:14.596638 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7de739-3c61-4e33-9f84-d5b29462eed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:14.597414 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7de739-3c61-4e33-9f84-d5b29462eed4 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:14.597693 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7de739-3c61-4e33-9f84-d5b29462eed4 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:14.597965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7de739-3c61-4e33-9f84-d5b29462eed4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:14.608310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7de739-3c61-4e33-9f84-d5b29462eed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:14.610482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7de739-3c61-4e33-9f84-d5b29462eed4 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-6e7de739-3c61-4e33-9f84-d5b29462eed4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:44:14.627054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:14.628118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:14.629390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:44:14.629390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:44:14.629390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:14.629390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:44:14.629390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 643/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:44:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:14.633761 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833c991-8cd4-4089-932e-7e6531d8fc1a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:14.731596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833c991-8cd4-4089-932e-7e6531d8fc1a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:14.764763 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0488a-0be0-4898-91bc-94dd9e2d9d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:14.765413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0488a-0be0-4898-91bc-94dd9e2d9d77 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:14.766282 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0488a-0be0-4898-91bc-94dd9e2d9d77 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:14.766594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0488a-0be0-4898-91bc-94dd9e2d9d77 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:14.767407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0488a-0be0-4898-91bc-94dd9e2d9d77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:14.767715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0488a-0be0-4898-91bc-94dd9e2d9d77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:14.769077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0488a-0be0-4898-91bc-94dd9e2d9d77 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:14.779929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 627/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:14.785534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7de739-3c61-4e33-9f84-d5b29462eed4 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:14.816367 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:14.881787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7de739-3c61-4e33-9f84-d5b29462eed4 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:14.913853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29b807-0256-421a-aeb1-b09669baab68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:14.914619 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29b807-0256-421a-aeb1-b09669baab68 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:14.914932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29b807-0256-421a-aeb1-b09669baab68 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:14.915201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29b807-0256-421a-aeb1-b09669baab68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:14.921597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29b807-0256-421a-aeb1-b09669baab68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:14.924818 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29b807-0256-421a-aeb1-b09669baab68 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-5d29b807-0256-421a-aeb1-b09669baab68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:44:14.939052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:14.940099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:14.941060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:44:14.941060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:44:14.941060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:44:14.941060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:44:14.941060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:44:14.941060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:44:14.941060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:44:14.941060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:44:14.941060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 644/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:44:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:14.945893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0488a-0be0-4898-91bc-94dd9e2d9d77 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:15.003252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:15.044315 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0488a-0be0-4898-91bc-94dd9e2d9d77 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:15.076283 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82b637-ddd8-41c9-863c-3f6f46b1f209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:15.076865 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82b637-ddd8-41c9-863c-3f6f46b1f209 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:15.077154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82b637-ddd8-41c9-863c-3f6f46b1f209 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:15.077432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82b637-ddd8-41c9-863c-3f6f46b1f209 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:15.078801 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82b637-ddd8-41c9-863c-3f6f46b1f209 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:15.079298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82b637-ddd8-41c9-863c-3f6f46b1f209 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:15.080714 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82b637-ddd8-41c9-863c-3f6f46b1f209 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:15.092199 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 628/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:44:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:15.098744 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29b807-0256-421a-aeb1-b09669baab68 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:15.195951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:15.205010 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29b807-0256-421a-aeb1-b09669baab68 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:15.238174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79921a7-9b3e-455e-8a3e-f3af6637fde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:15.239075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79921a7-9b3e-455e-8a3e-f3af6637fde4 None admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:15.239356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79921a7-9b3e-455e-8a3e-f3af6637fde4 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:15.239627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79921a7-9b3e-455e-8a3e-f3af6637fde4 None admin] PATH_INFO: `/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/b105f8163ce84a9ba725e6b04d348446/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:15.241548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79921a7-9b3e-455e-8a3e-f3af6637fde4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fc6fd42a31942688f570f1ce86be510, user_id=b105f8163ce84a9ba725e6b04d348446, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:15.242992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79921a7-9b3e-455e-8a3e-f3af6637fde4 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:15.255936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 645/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Thu May 19 13:44:15 2022] PUT /identity/v3/projects/2fc6fd42a31942688f570f1ce86be510/users/b105f8163ce84a9ba725e6b04d348446/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:44:17.482623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3a3da-9d42-4e41-8f5c-d634c757d338 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:17.483177 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3a3da-9d42-4e41-8f5c-d634c757d338 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:17.483271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3a3da-9d42-4e41-8f5c-d634c757d338 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:17.485092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 629/1274] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:44:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:44:17.490254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8b5f7-536d-4627-bceb-853b79e7f23e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:17.490649 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8b5f7-536d-4627-bceb-853b79e7f23e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:17.490927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8b5f7-536d-4627-bceb-853b79e7f23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:17.508242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e2b8b5f7-536d-4627-bceb-853b79e7f23e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:44:17.509412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e2b8b5f7-536d-4627-bceb-853b79e7f23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:44:17.573187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 646/1275] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:44:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:44:17.596226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3a3da-9d42-4e41-8f5c-d634c757d338 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:17.694854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3a3da-9d42-4e41-8f5c-d634c757d338 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:17.711254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:17.728482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28027637-5e46-46e5-be90-cefcf7149375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:17.729265 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28027637-5e46-46e5-be90-cefcf7149375 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:17.729552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28027637-5e46-46e5-be90-cefcf7149375 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:17.729829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28027637-5e46-46e5-be90-cefcf7149375 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:17.740249 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28027637-5e46-46e5-be90-cefcf7149375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-28027637-5e46-46e5-be90-cefcf7149375 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:44:17.747864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:17.749221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:17.750979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:17.750979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:17.750979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:17.750979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:17.750979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:44:17.750979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:44:17.750979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:44:17.750979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:44:17.750979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:44:17.750979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:44:17.750979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:44:17.750979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:17.757630 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 630/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Thu May 19 13:44:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:17.757787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8b5f7-536d-4627-bceb-853b79e7f23e None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:17.834992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:17.853944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8b5f7-536d-4627-bceb-853b79e7f23e None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:17.888390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de0fa9-44e6-4b7a-8cd3-77845f3ee198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:17.889033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de0fa9-44e6-4b7a-8cd3-77845f3ee198 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:17.889317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de0fa9-44e6-4b7a-8cd3-77845f3ee198 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:17.889595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de0fa9-44e6-4b7a-8cd3-77845f3ee198 None admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:17.890334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de0fa9-44e6-4b7a-8cd3-77845f3ee198 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:17.890715 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de0fa9-44e6-4b7a-8cd3-77845f3ee198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:17.892116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de0fa9-44e6-4b7a-8cd3-77845f3ee198 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:17.899987 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 647/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Thu May 19 13:44:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:17.905509 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28027637-5e46-46e5-be90-cefcf7149375 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:18.003422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28027637-5e46-46e5-be90-cefcf7149375 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:18.036412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a37439-e015-4103-af9d-1f5d2c71735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:18.037177 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a37439-e015-4103-af9d-1f5d2c71735f None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:18.037461 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a37439-e015-4103-af9d-1f5d2c71735f None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:18.037735 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a37439-e015-4103-af9d-1f5d2c71735f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:18.048139 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a37439-e015-4103-af9d-1f5d2c71735f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:18.049553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a37439-e015-4103-af9d-1f5d2c71735f None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-a8a37439-e015-4103-af9d-1f5d2c71735f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:44:18.066180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:18.067250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:18.068340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:44:18.068340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:44:18.068340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:18.068340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:18.068340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 631/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:44:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:18.072885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de0fa9-44e6-4b7a-8cd3-77845f3ee198 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:18.176424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de0fa9-44e6-4b7a-8cd3-77845f3ee198 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:18.210576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7898dd-333e-4e46-a3fd-275fccb1895b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:18.211259 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7898dd-333e-4e46-a3fd-275fccb1895b None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:18.211542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7898dd-333e-4e46-a3fd-275fccb1895b None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:18.211817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7898dd-333e-4e46-a3fd-275fccb1895b None admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:18.212528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7898dd-333e-4e46-a3fd-275fccb1895b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:18.212832 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7898dd-333e-4e46-a3fd-275fccb1895b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:18.214179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7898dd-333e-4e46-a3fd-275fccb1895b None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:18.226233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 648/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:18.231915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a37439-e015-4103-af9d-1f5d2c71735f None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:18.261727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:18.330190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a37439-e015-4103-af9d-1f5d2c71735f None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:18.362055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64112f08-ffee-419c-af87-1e1de13393e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:18.362826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64112f08-ffee-419c-af87-1e1de13393e2 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:18.363139 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64112f08-ffee-419c-af87-1e1de13393e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:18.363414 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64112f08-ffee-419c-af87-1e1de13393e2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:18.371324 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64112f08-ffee-419c-af87-1e1de13393e2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:18.372762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64112f08-ffee-419c-af87-1e1de13393e2 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-64112f08-ffee-419c-af87-1e1de13393e2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:44:18.387426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:18.388562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:18.394891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:18.394891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:18.394891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:44:18.394891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:44:18.394891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:44:18.394891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:44:18.394891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:44:18.394891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:44:18.394891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 13:44:18.394891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:18.394891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 632/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:44:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:18.395532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7898dd-333e-4e46-a3fd-275fccb1895b None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:18.399190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:18.493943 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7898dd-333e-4e46-a3fd-275fccb1895b None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:18.530630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152c56d-88d6-47b5-b5e1-226e27ff5d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:18.531281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152c56d-88d6-47b5-b5e1-226e27ff5d00 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:18.531568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152c56d-88d6-47b5-b5e1-226e27ff5d00 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:18.531848 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152c56d-88d6-47b5-b5e1-226e27ff5d00 None admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:18.532684 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152c56d-88d6-47b5-b5e1-226e27ff5d00 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:18.532997 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152c56d-88d6-47b5-b5e1-226e27ff5d00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:18.534542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152c56d-88d6-47b5-b5e1-226e27ff5d00 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:18.551304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 649/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Thu May 19 13:44:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:18.557148 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64112f08-ffee-419c-af87-1e1de13393e2 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:18.646351 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:18.655864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64112f08-ffee-419c-af87-1e1de13393e2 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:18.688607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafd4c1-c55c-485c-a6c6-31ebb9467ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:18.689235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafd4c1-c55c-485c-a6c6-31ebb9467ae6 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:18.689516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafd4c1-c55c-485c-a6c6-31ebb9467ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:18.689789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafd4c1-c55c-485c-a6c6-31ebb9467ae6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:18.754004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafd4c1-c55c-485c-a6c6-31ebb9467ae6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:18.755488 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafd4c1-c55c-485c-a6c6-31ebb9467ae6 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:18.839248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:18.849790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 633/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Thu May 19 13:44:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:44:18.855689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152c56d-88d6-47b5-b5e1-226e27ff5d00 None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:18.915311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:18.954569 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152c56d-88d6-47b5-b5e1-226e27ff5d00 None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:18.987410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d1dc1-7456-493a-b63f-014acfbed1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:18.988184 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d1dc1-7456-493a-b63f-014acfbed1ff None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:18.988465 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d1dc1-7456-493a-b63f-014acfbed1ff None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:18.988735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d1dc1-7456-493a-b63f-014acfbed1ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:18.998627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d1dc1-7456-493a-b63f-014acfbed1ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:19.000780 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d1dc1-7456-493a-b63f-014acfbed1ff None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-2d9d1dc1-7456-493a-b63f-014acfbed1ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:44:19.017498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:19.018631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:19.019925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:44:19.019925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:44:19.019925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:44:19.019925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:44:19.019925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 650/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:19.024297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafd4c1-c55c-485c-a6c6-31ebb9467ae6 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:19.120954 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafd4c1-c55c-485c-a6c6-31ebb9467ae6 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:19.154644 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea671c4-4803-4dd4-9e9e-591507e5a698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:19.155322 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea671c4-4803-4dd4-9e9e-591507e5a698 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:19.155607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea671c4-4803-4dd4-9e9e-591507e5a698 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:19.155882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea671c4-4803-4dd4-9e9e-591507e5a698 None admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:19.157309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea671c4-4803-4dd4-9e9e-591507e5a698 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:19.157614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea671c4-4803-4dd4-9e9e-591507e5a698 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:19.159038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea671c4-4803-4dd4-9e9e-591507e5a698 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:19.170154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 634/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Thu May 19 13:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:19.176983 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d1dc1-7456-493a-b63f-014acfbed1ff None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:19.277186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d1dc1-7456-493a-b63f-014acfbed1ff None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:19.291585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:19.311427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096525e4-b926-49f8-9275-db149e2941dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:19.312208 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096525e4-b926-49f8-9275-db149e2941dc None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:19.313016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096525e4-b926-49f8-9275-db149e2941dc None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:19.313368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096525e4-b926-49f8-9275-db149e2941dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:19.319797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096525e4-b926-49f8-9275-db149e2941dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:19.321228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096525e4-b926-49f8-9275-db149e2941dc None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application [None req-096525e4-b926-49f8-9275-db149e2941dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:44:19.335395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application with Lock( May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:19.336440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:19.337718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:44:19.337718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:44:19.337718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:44:19.337718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:44:19.337718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:44:19.337718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:44:19.337718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:44:19.337718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: ERROR keystone.server.flask.application May 19 13:44:19.337718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 651/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Thu May 19 13:44:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:19.342368 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea671c4-4803-4dd4-9e9e-591507e5a698 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:19.391571 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:19.441491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea671c4-4803-4dd4-9e9e-591507e5a698 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:19.474112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe7401-8f97-443a-9568-9431b4ab4da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:19.474764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe7401-8f97-443a-9568-9431b4ab4da2 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:19.475076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe7401-8f97-443a-9568-9431b4ab4da2 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:19.475355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe7401-8f97-443a-9568-9431b4ab4da2 None admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:19.476135 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe7401-8f97-443a-9568-9431b4ab4da2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:19.476444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe7401-8f97-443a-9568-9431b4ab4da2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:19.477858 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe7401-8f97-443a-9568-9431b4ab4da2 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:19.490024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 635/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Thu May 19 13:44:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:19.496004 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096525e4-b926-49f8-9275-db149e2941dc None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:19.594717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096525e4-b926-49f8-9275-db149e2941dc None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:19.626603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd5b91-d3bd-49f3-87fa-56efa1d40fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:19.627273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd5b91-d3bd-49f3-87fa-56efa1d40fa9 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:19.627558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd5b91-d3bd-49f3-87fa-56efa1d40fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:19.627833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd5b91-d3bd-49f3-87fa-56efa1d40fa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:19.628654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd5b91-d3bd-49f3-87fa-56efa1d40fa9 None admin] RBAC: Adding query filter params (role.id=0893bb944dd94634accd3906aa391cd2, scope.project.id=2fc6fd42a31942688f570f1ce86be510, user.id=b105f8163ce84a9ba725e6b04d348446) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:19.628966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd5b91-d3bd-49f3-87fa-56efa1d40fa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:19.630384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd5b91-d3bd-49f3-87fa-56efa1d40fa9 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:19.636087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 652/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Thu May 19 13:44:19 2022] GET /identity/v3/role_assignments?user.id=b105f8163ce84a9ba725e6b04d348446&scope.project.id=2fc6fd42a31942688f570f1ce86be510&role.id=0893bb944dd94634accd3906aa391cd2 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:21.950166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d4072-192a-45ed-8b62-f5e5685b82b2 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:21.950706 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d4072-192a-45ed-8b62-f5e5685b82b2 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:21.950803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d4072-192a-45ed-8b62-f5e5685b82b2 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:21.952157 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 636/1288] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:44:21.957442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb8df4-07f0-4cc0-92a2-a469f6ba9432 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:21.957844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb8df4-07f0-4cc0-92a2-a469f6ba9432 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:21.958005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb8df4-07f0-4cc0-92a2-a469f6ba9432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:21.975009 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-c8bb8df4-07f0-4cc0-92a2-a469f6ba9432 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:44:21.976199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c8bb8df4-07f0-4cc0-92a2-a469f6ba9432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:44:22.006738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:22.043680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 653/1289] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:44:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:44:22.066173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d4072-192a-45ed-8b62-f5e5685b82b2 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:22.179730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d4072-192a-45ed-8b62-f5e5685b82b2 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:22.194213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:22.218049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ffce3-b2eb-4e77-9b17-2b8b404e95a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:22.219812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ffce3-b2eb-4e77-9b17-2b8b404e95a4 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:22.220098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ffce3-b2eb-4e77-9b17-2b8b404e95a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:22.220376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ffce3-b2eb-4e77-9b17-2b8b404e95a4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:22.221024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ffce3-b2eb-4e77-9b17-2b8b404e95a4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:22.222380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ffce3-b2eb-4e77-9b17-2b8b404e95a4 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-735ffce3-b2eb-4e77-9b17-2b8b404e95a4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:22.233901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:22.235986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:22.237116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 637/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Thu May 19 13:44:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:22.242303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb8df4-07f0-4cc0-92a2-a469f6ba9432 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:22.337335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb8df4-07f0-4cc0-92a2-a469f6ba9432 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:22.370409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a585ce-e493-46d9-9a00-2a9cde73b56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:22.371086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a585ce-e493-46d9-9a00-2a9cde73b56a None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:22.371378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a585ce-e493-46d9-9a00-2a9cde73b56a None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:22.371653 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a585ce-e493-46d9-9a00-2a9cde73b56a None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:22.372351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a585ce-e493-46d9-9a00-2a9cde73b56a None admin] RBAC: Adding query filter params (name=placement) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:22.372656 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a585ce-e493-46d9-9a00-2a9cde73b56a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:22.374262 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a585ce-e493-46d9-9a00-2a9cde73b56a None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:22.379236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 654/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Thu May 19 13:44:22 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:22.384788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ffce3-b2eb-4e77-9b17-2b8b404e95a4 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:22.482596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ffce3-b2eb-4e77-9b17-2b8b404e95a4 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:22.515003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae481ab7-f521-4332-a294-bf784f94066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:22.516267 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae481ab7-f521-4332-a294-bf784f94066e None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:22.516554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae481ab7-f521-4332-a294-bf784f94066e None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:22.516823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae481ab7-f521-4332-a294-bf784f94066e None admin] PATH_INFO: `/v3/services` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:22.517517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae481ab7-f521-4332-a294-bf784f94066e None admin] RBAC: Adding query filter params (type=placement) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:22.517820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae481ab7-f521-4332-a294-bf784f94066e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:22.519197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae481ab7-f521-4332-a294-bf784f94066e None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:22.523516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 638/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Thu May 19 13:44:22 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:24.698749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92dbb2-ae7a-48e6-b1e7-18d4687b4642 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:24.699279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92dbb2-ae7a-48e6-b1e7-18d4687b4642 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:24.699391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92dbb2-ae7a-48e6-b1e7-18d4687b4642 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:24.700718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 655/1293] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:44:24.706064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ced43-9c90-4b71-af90-4d9cb7b2fc35 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:24.706487 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ced43-9c90-4b71-af90-4d9cb7b2fc35 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:24.706635 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ced43-9c90-4b71-af90-4d9cb7b2fc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:24.723107 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-4c7ced43-9c90-4b71-af90-4d9cb7b2fc35 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:44:24.724307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-4c7ced43-9c90-4b71-af90-4d9cb7b2fc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:44:24.754417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:24.790549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 639/1294] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:44:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:44:24.813846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92dbb2-ae7a-48e6-b1e7-18d4687b4642 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:24.843686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:24.913111 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92dbb2-ae7a-48e6-b1e7-18d4687b4642 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:24.955239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f5208-5a42-465a-9f2b-62ebe406214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:24.956302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f5208-5a42-465a-9f2b-62ebe406214d None admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:24.956796 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f5208-5a42-465a-9f2b-62ebe406214d None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:24.957282 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f5208-5a42-465a-9f2b-62ebe406214d None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:24.958491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f5208-5a42-465a-9f2b-62ebe406214d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:24.959910 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f5208-5a42-465a-9f2b-62ebe406214d None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:24.973979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 656/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Thu May 19 13:44:24 2022] POST /identity/v3/services => generated 246 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:44:27.161832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3704b8-b492-4fc9-98fc-451975dad0ca None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:27.162396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3704b8-b492-4fc9-98fc-451975dad0ca None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:27.162396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3704b8-b492-4fc9-98fc-451975dad0ca None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:27.163782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 640/1296] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:44:27.168969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a79437-edaf-431b-827c-2711cd3b1ea2 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:27.169372 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a79437-edaf-431b-827c-2711cd3b1ea2 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:27.169526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a79437-edaf-431b-827c-2711cd3b1ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:27.186607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-19a79437-edaf-431b-827c-2711cd3b1ea2 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:44:27.188388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-19a79437-edaf-431b-827c-2711cd3b1ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:44:27.273478 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 657/1297] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:44:27 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:44:27.296152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3704b8-b492-4fc9-98fc-451975dad0ca None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:27.398089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3704b8-b492-4fc9-98fc-451975dad0ca None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:27.431588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2948741-db08-4816-9be0-960eb0973a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:27.432311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2948741-db08-4816-9be0-960eb0973a73 None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:27.432595 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2948741-db08-4816-9be0-960eb0973a73 None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:27.432867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2948741-db08-4816-9be0-960eb0973a73 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:27.433510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2948741-db08-4816-9be0-960eb0973a73 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:27.434874 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2948741-db08-4816-9be0-960eb0973a73 None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-a2948741-db08-4816-9be0-960eb0973a73 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:27.447293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:27.448445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:27.449306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 641/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Thu May 19 13:44:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:27.455189 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a79437-edaf-431b-827c-2711cd3b1ea2 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:27.472756 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:27.553094 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a79437-edaf-431b-827c-2711cd3b1ea2 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:27.586708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb00ba1-adb0-4aa8-9696-e6751fd538bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:27.587387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb00ba1-adb0-4aa8-9696-e6751fd538bf None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:27.587675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb00ba1-adb0-4aa8-9696-e6751fd538bf None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:27.587953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb00ba1-adb0-4aa8-9696-e6751fd538bf None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:27.588655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb00ba1-adb0-4aa8-9696-e6751fd538bf None admin] RBAC: Adding query filter params (name=placement) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:27.588960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb00ba1-adb0-4aa8-9696-e6751fd538bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:27.590320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb00ba1-adb0-4aa8-9696-e6751fd538bf None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:27.595411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 658/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Thu May 19 13:44:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:27.601279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2948741-db08-4816-9be0-960eb0973a73 None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:27.632576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:27.699147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2948741-db08-4816-9be0-960eb0973a73 None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:27.730804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944f654-8085-42c6-9500-0af2c6e23ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:27.731479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944f654-8085-42c6-9500-0af2c6e23ddd None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:27.731764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944f654-8085-42c6-9500-0af2c6e23ddd None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:27.732038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944f654-8085-42c6-9500-0af2c6e23ddd None admin] PATH_INFO: `/v3/endpoints` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:27.732822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944f654-8085-42c6-9500-0af2c6e23ddd None admin] RBAC: Adding query filter params (interface=public, service_id=0d8f47401dd0480480c8add81b8d88e1, region_id=RegionOne) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:27.733136 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944f654-8085-42c6-9500-0af2c6e23ddd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:27.735066 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944f654-8085-42c6-9500-0af2c6e23ddd None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:27.739966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 642/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Thu May 19 13:44:27 2022] GET /identity/v3/endpoints?service_id=0d8f47401dd0480480c8add81b8d88e1&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:27.745562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb00ba1-adb0-4aa8-9696-e6751fd538bf None admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:27.824558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:27.841986 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb00ba1-adb0-4aa8-9696-e6751fd538bf None admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:27.875691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123fdea-ccfa-4444-8abc-464410a94c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:27.876339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123fdea-ccfa-4444-8abc-464410a94c23 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:27.876625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123fdea-ccfa-4444-8abc-464410a94c23 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:27.876904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123fdea-ccfa-4444-8abc-464410a94c23 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:27.877576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123fdea-ccfa-4444-8abc-464410a94c23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:27.878951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123fdea-ccfa-4444-8abc-464410a94c23 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:27.885581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 659/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Thu May 19 13:44:27 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:44:30.076492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42b642-579d-4235-a9f2-41c45e95759e None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:30.077042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42b642-579d-4235-a9f2-41c45e95759e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:30.077042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42b642-579d-4235-a9f2-41c45e95759e None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:30.078969 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 643/1302] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:44:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:44:30.084196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495fe92-08fa-4e70-9a0d-41fa6363fe64 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:30.084586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495fe92-08fa-4e70-9a0d-41fa6363fe64 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:30.084753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495fe92-08fa-4e70-9a0d-41fa6363fe64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:30.101679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-d495fe92-08fa-4e70-9a0d-41fa6363fe64 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:44:30.102885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d495fe92-08fa-4e70-9a0d-41fa6363fe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:44:30.167262 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 660/1303] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:44:30 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:44:30.190751 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42b642-579d-4235-a9f2-41c45e95759e None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:30.283217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:30.290088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42b642-579d-4235-a9f2-41c45e95759e None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:30.322796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda8a1f-ded9-4c46-8302-ecbd159b3b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:30.323605 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda8a1f-ded9-4c46-8302-ecbd159b3b6a None admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:30.323885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda8a1f-ded9-4c46-8302-ecbd159b3b6a None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:30.324157 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda8a1f-ded9-4c46-8302-ecbd159b3b6a None admin] PATH_INFO: `/v3/services/placement` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:30.324794 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda8a1f-ded9-4c46-8302-ecbd159b3b6a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:30.326129 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda8a1f-ded9-4c46-8302-ecbd159b3b6a None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application [None req-fcda8a1f-ded9-4c46-8302-ecbd159b3b6a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:44:30.338009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:44:30.339204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application with Lock( May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self._enter() May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application return self.creator() May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application created_value = creator( May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: ERROR keystone.server.flask.application May 19 13:44:30.340057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 644/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Thu May 19 13:44:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:44:30.345781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495fe92-08fa-4e70-9a0d-41fa6363fe64 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:30.420950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:30.443518 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495fe92-08fa-4e70-9a0d-41fa6363fe64 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:30.476009 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8660b-b2ea-4482-87b3-a61951772864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:30.476659 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8660b-b2ea-4482-87b3-a61951772864 None admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:30.476941 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8660b-b2ea-4482-87b3-a61951772864 None admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:30.477212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8660b-b2ea-4482-87b3-a61951772864 None admin] PATH_INFO: `/v3/services` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:30.477908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8660b-b2ea-4482-87b3-a61951772864 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:44:30.478210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8660b-b2ea-4482-87b3-a61951772864 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:30.479586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8660b-b2ea-4482-87b3-a61951772864 None admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:30.485089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 661/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Thu May 19 13:44:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:44:30.490825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda8a1f-ded9-4c46-8302-ecbd159b3b6a None admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:30.588005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda8a1f-ded9-4c46-8302-ecbd159b3b6a None admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:30.621031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfc57e-4b00-4eaa-8733-5043723e9eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:30.621673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfc57e-4b00-4eaa-8733-5043723e9eca None admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:30.621955 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfc57e-4b00-4eaa-8733-5043723e9eca None admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:30.622227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfc57e-4b00-4eaa-8733-5043723e9eca None admin] PATH_INFO: `/v3/endpoints` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:30.622988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfc57e-4b00-4eaa-8733-5043723e9eca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:30.624633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfc57e-4b00-4eaa-8733-5043723e9eca None admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:30.641715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 645/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Thu May 19 13:44:30 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:44:40.481628 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=30 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:40.637511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:34.395158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0b710-0440-4bb1-bd08-6467267bc202 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.396061 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0b710-0440-4bb1-bd08-6467267bc202 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.396493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0b710-0440-4bb1-bd08-6467267bc202 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.401989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 662/1307] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:45:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:45:34.403456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb826f-84ef-4511-b9ce-e2d8e5e3bddf None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.404888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb826f-84ef-4511-b9ce-e2d8e5e3bddf None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.405789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb826f-84ef-4511-b9ce-e2d8e5e3bddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.513311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-1acb826f-84ef-4511-b9ce-e2d8e5e3bddf None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:34.514982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1acb826f-84ef-4511-b9ce-e2d8e5e3bddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:34.701861 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 646/1308] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:45:34.739585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb420122-cf7d-4af7-9158-f5a9aa079ffa None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.739876 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb420122-cf7d-4af7-9158-f5a9aa079ffa None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.740157 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb420122-cf7d-4af7-9158-f5a9aa079ffa None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.741892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 663/1309] 10.222.0.52 () {54 vars in 964 bytes} [Thu May 19 13:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:45:34.747798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800513a-851b-45f9-bbc6-6080acdaff48 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.749079 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800513a-851b-45f9-bbc6-6080acdaff48 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.749734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800513a-851b-45f9-bbc6-6080acdaff48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.852445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-b800513a-851b-45f9-bbc6-6080acdaff48 None None] MFA Rules not processed for user `f18f635bff0142409e90da66547927f1`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:34.853687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b800513a-851b-45f9-bbc6-6080acdaff48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:34.876221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:34.937358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 647/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Thu May 19 13:45:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:34.946311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb420122-cf7d-4af7-9158-f5a9aa079ffa None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:34.948428 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-fb420122-cf7d-4af7-9158-f5a9aa079ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:35.061894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb420122-cf7d-4af7-9158-f5a9aa079ffa None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:35.141351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224314e-280b-4c0e-a717-5b00ccd2967e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:35.142633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224314e-280b-4c0e-a717-5b00ccd2967e service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:35.142961 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224314e-280b-4c0e-a717-5b00ccd2967e service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:35.143246 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224314e-280b-4c0e-a717-5b00ccd2967e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:35.144902 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-6224314e-280b-4c0e-a717-5b00ccd2967e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:35.205644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:35.265229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224314e-280b-4c0e-a717-5b00ccd2967e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:35.266679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224314e-280b-4c0e-a717-5b00ccd2967e service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:35.286377 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 664/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:45:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:45:37.869105 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52749c93-ef9a-4f9e-918e-aa8406cb5924 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:37.869684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52749c93-ef9a-4f9e-918e-aa8406cb5924 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:37.869769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52749c93-ef9a-4f9e-918e-aa8406cb5924 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:37.871579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 648/1312] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:45:37.877467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f7ed2-a42c-497d-af47-0c608fcdfa8a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:37.877949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f7ed2-a42c-497d-af47-0c608fcdfa8a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:37.878050 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f7ed2-a42c-497d-af47-0c608fcdfa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:37.953724 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-8a1f7ed2-a42c-497d-af47-0c608fcdfa8a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:37.955044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8a1f7ed2-a42c-497d-af47-0c608fcdfa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:38.059013 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 665/1313] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:38.094586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52749c93-ef9a-4f9e-918e-aa8406cb5924 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:38.096261 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-52749c93-ef9a-4f9e-918e-aa8406cb5924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:38.209815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52749c93-ef9a-4f9e-918e-aa8406cb5924 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:38.304853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:38.308111 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50845117-a395-42ee-ad8a-2397d546c128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:38.308763 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50845117-a395-42ee-ad8a-2397d546c128 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:38.309693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50845117-a395-42ee-ad8a-2397d546c128 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:38.309972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50845117-a395-42ee-ad8a-2397d546c128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:38.313599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-50845117-a395-42ee-ad8a-2397d546c128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:38.434209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50845117-a395-42ee-ad8a-2397d546c128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:38.436791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50845117-a395-42ee-ad8a-2397d546c128 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:38.456447 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 649/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:45:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:45:41.048570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c8556-4bfd-47a1-92ea-3e1da028d4d8 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:41.049152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c8556-4bfd-47a1-92ea-3e1da028d4d8 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:41.049152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c8556-4bfd-47a1-92ea-3e1da028d4d8 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:41.050505 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 666/1315] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:45:41.055790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9909f27-29fc-4e18-b508-8879ce4aeb79 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:41.056169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9909f27-29fc-4e18-b508-8879ce4aeb79 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:41.056345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9909f27-29fc-4e18-b508-8879ce4aeb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:41.118955 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a9909f27-29fc-4e18-b508-8879ce4aeb79 None None] MFA Rules not processed for user `3bd06221895d4193813d7412e1e6a315`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:41.120164 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a9909f27-29fc-4e18-b508-8879ce4aeb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:41.172230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:41.215866 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 650/1316] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:45:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:41.255062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c8556-4bfd-47a1-92ea-3e1da028d4d8 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:41.256577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e02c8556-4bfd-47a1-92ea-3e1da028d4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:41.267133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:41.359347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c8556-4bfd-47a1-92ea-3e1da028d4d8 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:41.432451 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b11ba5-8c0b-43db-84e4-fb32a741d117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:41.433101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b11ba5-8c0b-43db-84e4-fb32a741d117 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:41.433384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b11ba5-8c0b-43db-84e4-fb32a741d117 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:41.433661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b11ba5-8c0b-43db-84e4-fb32a741d117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:41.435323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-28b11ba5-8c0b-43db-84e4-fb32a741d117 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:41.496970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:41.542744 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b11ba5-8c0b-43db-84e4-fb32a741d117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:41.544196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b11ba5-8c0b-43db-84e4-fb32a741d117 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:41.562504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 667/1317] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:45:41 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:45:44.352612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba519364-5db6-4a85-aa3d-36328aed9d7d None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:44.354489 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba519364-5db6-4a85-aa3d-36328aed9d7d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:44.354489 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba519364-5db6-4a85-aa3d-36328aed9d7d None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:44.354681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 651/1318] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:45:44.360113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a5a52-1eb8-4a12-a7bc-4704910c4a03 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:44.360526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a5a52-1eb8-4a12-a7bc-4704910c4a03 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:44.360679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a5a52-1eb8-4a12-a7bc-4704910c4a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:44.424150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-1e0a5a52-1eb8-4a12-a7bc-4704910c4a03 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:44.425376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1e0a5a52-1eb8-4a12-a7bc-4704910c4a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:44.522222 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 668/1319] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:45:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:44.577011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd3ae2-90d0-4b79-91f2-895beb1195a1 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:44.577352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd3ae2-90d0-4b79-91f2-895beb1195a1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:44.577563 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd3ae2-90d0-4b79-91f2-895beb1195a1 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:44.578953 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 652/1320] 10.222.0.52 () {54 vars in 964 bytes} [Thu May 19 13:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:45:44.584909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93324197-cebc-4a00-ba42-aeb105c5fecd None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:44.585203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93324197-cebc-4a00-ba42-aeb105c5fecd None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:44.585483 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93324197-cebc-4a00-ba42-aeb105c5fecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:44.605874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:44.667179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-93324197-cebc-4a00-ba42-aeb105c5fecd None None] MFA Rules not processed for user `f18f635bff0142409e90da66547927f1`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:44.668376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-93324197-cebc-4a00-ba42-aeb105c5fecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:44.742373 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 669/1321] 10.222.0.52 () {58 vars in 1061 bytes} [Thu May 19 13:45:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:44.752154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd3ae2-90d0-4b79-91f2-895beb1195a1 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:44.753672 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-33bd3ae2-90d0-4b79-91f2-895beb1195a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:44.855145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd3ae2-90d0-4b79-91f2-895beb1195a1 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:44.927130 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370270d1-4f22-4581-9430-1f2244812d77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:44.927775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370270d1-4f22-4581-9430-1f2244812d77 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:44.928059 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370270d1-4f22-4581-9430-1f2244812d77 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:44.928333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370270d1-4f22-4581-9430-1f2244812d77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:44.929962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-370270d1-4f22-4581-9430-1f2244812d77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:44.937828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:45.039697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370270d1-4f22-4581-9430-1f2244812d77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:45.041127 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370270d1-4f22-4581-9430-1f2244812d77 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:45.059778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 653/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:45:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:45:48.401067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a8ac1-a96f-4f2c-98f6-7bce664e705e None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:48.401481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a8ac1-a96f-4f2c-98f6-7bce664e705e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:48.401624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a8ac1-a96f-4f2c-98f6-7bce664e705e None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:48.403028 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 670/1323] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:45:48.408966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d65b0-8943-432d-9cbc-5391967d1a97 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:48.409425 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d65b0-8943-432d-9cbc-5391967d1a97 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:48.409536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d65b0-8943-432d-9cbc-5391967d1a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:48.475217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-c15d65b0-8943-432d-9cbc-5391967d1a97 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:48.476437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c15d65b0-8943-432d-9cbc-5391967d1a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:48.533737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:48.573433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 654/1324] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:48.611243 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a8ac1-a96f-4f2c-98f6-7bce664e705e None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:48.612798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-bc0a8ac1-a96f-4f2c-98f6-7bce664e705e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:48.721547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a8ac1-a96f-4f2c-98f6-7bce664e705e None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:48.734353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:48.799521 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53081b-ab9e-4f67-b71a-6d858889063b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:48.800167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53081b-ab9e-4f67-b71a-6d858889063b service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:48.800447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53081b-ab9e-4f67-b71a-6d858889063b service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:48.800719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53081b-ab9e-4f67-b71a-6d858889063b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:48.802343 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ab53081b-ab9e-4f67-b71a-6d858889063b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:48.912735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53081b-ab9e-4f67-b71a-6d858889063b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:48.914179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53081b-ab9e-4f67-b71a-6d858889063b service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:48.932295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 671/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:45:51.239384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84d817-6516-414e-b909-564b16c33d5d None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:51.239977 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84d817-6516-414e-b909-564b16c33d5d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:51.239977 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84d817-6516-414e-b909-564b16c33d5d None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:51.241288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 655/1326] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:45:51.246491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62986f-4491-4f57-b62c-b996f418e0b6 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:51.246905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62986f-4491-4f57-b62c-b996f418e0b6 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:51.247078 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62986f-4491-4f57-b62c-b996f418e0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:51.295697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:51.314616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-cb62986f-4491-4f57-b62c-b996f418e0b6 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:51.315895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-cb62986f-4491-4f57-b62c-b996f418e0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:51.415977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 672/1327] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:51.452298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84d817-6516-414e-b909-564b16c33d5d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:51.453896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-bc84d817-6516-414e-b909-564b16c33d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:51.555001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84d817-6516-414e-b909-564b16c33d5d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:51.629785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0ff70-dc50-4b4f-9974-46314e49c871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:51.630427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0ff70-dc50-4b4f-9974-46314e49c871 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:51.630709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0ff70-dc50-4b4f-9974-46314e49c871 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:51.631028 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0ff70-dc50-4b4f-9974-46314e49c871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:51.632633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1bf0ff70-dc50-4b4f-9974-46314e49c871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:51.650107 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:51.753765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0ff70-dc50-4b4f-9974-46314e49c871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:51.755246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0ff70-dc50-4b4f-9974-46314e49c871 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:51.773647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 656/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:45:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:45:54.118118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664169a-c866-464b-8204-0d9ca25113fb None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:54.118678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664169a-c866-464b-8204-0d9ca25113fb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:54.118678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664169a-c866-464b-8204-0d9ca25113fb None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:54.120215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 673/1329] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:45:54.125694 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48a5f9-9a3d-41ba-a9aa-8c0f69ca1d7f None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:54.126107 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48a5f9-9a3d-41ba-a9aa-8c0f69ca1d7f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:54.126250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48a5f9-9a3d-41ba-a9aa-8c0f69ca1d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:54.191774 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3e48a5f9-9a3d-41ba-a9aa-8c0f69ca1d7f None None] MFA Rules not processed for user `3bd06221895d4193813d7412e1e6a315`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:54.192979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3e48a5f9-9a3d-41ba-a9aa-8c0f69ca1d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:54.260338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:54.293899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 657/1330] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:45:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:54.331888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664169a-c866-464b-8204-0d9ca25113fb None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:54.333430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4664169a-c866-464b-8204-0d9ca25113fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:54.442529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664169a-c866-464b-8204-0d9ca25113fb None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:54.464030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:54.523054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed55c1-4f52-4f4b-889e-3cb966090720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:54.523698 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed55c1-4f52-4f4b-889e-3cb966090720 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:54.523976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed55c1-4f52-4f4b-889e-3cb966090720 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:54.524247 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed55c1-4f52-4f4b-889e-3cb966090720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:54.525870 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-29ed55c1-4f52-4f4b-889e-3cb966090720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:54.639558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed55c1-4f52-4f4b-889e-3cb966090720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:54.641344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed55c1-4f52-4f4b-889e-3cb966090720 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:54.661141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 674/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:45:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:45:57.126035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73594b-49fd-47f6-9721-a6295ff8f777 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:57.126569 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73594b-49fd-47f6-9721-a6295ff8f777 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:57.126569 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73594b-49fd-47f6-9721-a6295ff8f777 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:57.128024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 658/1332] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:45:57.133612 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd64e69-a5d2-476c-a547-333f8c42cf6f None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:57.134039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd64e69-a5d2-476c-a547-333f8c42cf6f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:57.134293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd64e69-a5d2-476c-a547-333f8c42cf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:57.195493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:57.198496 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-5bd64e69-a5d2-476c-a547-333f8c42cf6f None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:57.200222 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5bd64e69-a5d2-476c-a547-333f8c42cf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:57.300233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 675/1333] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:57.334846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73594b-49fd-47f6-9721-a6295ff8f777 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:57.336412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ae73594b-49fd-47f6-9721-a6295ff8f777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:57.440263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73594b-49fd-47f6-9721-a6295ff8f777 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:57.512751 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23d781-5b84-4665-a2df-41aae7b23058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:57.513324 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23d781-5b84-4665-a2df-41aae7b23058 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:57.513612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23d781-5b84-4665-a2df-41aae7b23058 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:57.513891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23d781-5b84-4665-a2df-41aae7b23058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:57.515583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1f23d781-5b84-4665-a2df-41aae7b23058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:57.542684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:57.626211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23d781-5b84-4665-a2df-41aae7b23058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:57.627680 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23d781-5b84-4665-a2df-41aae7b23058 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:57.646704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 659/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:45:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:45:59.917482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55aaad8-7c97-40be-9bbf-0b6a244cf23b None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:59.918046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55aaad8-7c97-40be-9bbf-0b6a244cf23b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:59.918046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55aaad8-7c97-40be-9bbf-0b6a244cf23b None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:59.920097 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 676/1335] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:45:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:45:59.925247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a2367-1816-4335-9008-8aa992159a55 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:59.925689 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a2367-1816-4335-9008-8aa992159a55 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:59.925811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a2367-1816-4335-9008-8aa992159a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:59.989754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-1b3a2367-1816-4335-9008-8aa992159a55 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:59.990992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1b3a2367-1816-4335-9008-8aa992159a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:00.063945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:00.086643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 660/1336] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:00.126149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55aaad8-7c97-40be-9bbf-0b6a244cf23b None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:00.127717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d55aaad8-7c97-40be-9bbf-0b6a244cf23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:00.229164 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55aaad8-7c97-40be-9bbf-0b6a244cf23b None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:00.256604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:00.301404 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3daf2d-1497-4107-b8ef-014dbf68a19e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:00.302046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3daf2d-1497-4107-b8ef-014dbf68a19e service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:00.302331 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3daf2d-1497-4107-b8ef-014dbf68a19e service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:00.302610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3daf2d-1497-4107-b8ef-014dbf68a19e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:00.304361 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9e3daf2d-1497-4107-b8ef-014dbf68a19e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:00.414531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3daf2d-1497-4107-b8ef-014dbf68a19e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:00.416012 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3daf2d-1497-4107-b8ef-014dbf68a19e service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:00.434166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 677/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:03.008712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adfd86-4540-42a7-ac95-52568c019cff None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.009285 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adfd86-4540-42a7-ac95-52568c019cff None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.009285 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adfd86-4540-42a7-ac95-52568c019cff None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.010626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 661/1338] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:03.015975 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4582710-94fb-4f47-98de-93837b25274e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.016363 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4582710-94fb-4f47-98de-93837b25274e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.016547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4582710-94fb-4f47-98de-93837b25274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.080806 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-a4582710-94fb-4f47-98de-93837b25274e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:03.082017 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a4582710-94fb-4f47-98de-93837b25274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:03.090268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:03.182349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 678/1339] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:03.218434 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adfd86-4540-42a7-ac95-52568c019cff None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:03.220010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-57adfd86-4540-42a7-ac95-52568c019cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:03.323067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adfd86-4540-42a7-ac95-52568c019cff None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:03.394779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2c71b-264c-4ba9-bc60-8257c16829b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:03.395463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2c71b-264c-4ba9-bc60-8257c16829b9 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.395749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2c71b-264c-4ba9-bc60-8257c16829b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.396022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2c71b-264c-4ba9-bc60-8257c16829b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.397668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-52f2c71b-264c-4ba9-bc60-8257c16829b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:03.433458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:03.505619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2c71b-264c-4ba9-bc60-8257c16829b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:03.507080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2c71b-264c-4ba9-bc60-8257c16829b9 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:03.526015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 662/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:07.968068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2953b7-bfe8-4b13-953f-1d73064159d1 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:07.968626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2953b7-bfe8-4b13-953f-1d73064159d1 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:07.968626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2953b7-bfe8-4b13-953f-1d73064159d1 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:07.970062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 679/1341] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:07.975431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9015e-78d8-463d-8ca4-8d289b64af1b None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:07.975882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9015e-78d8-463d-8ca4-8d289b64af1b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:07.976136 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9015e-78d8-463d-8ca4-8d289b64af1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:08.046062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3aa9015e-78d8-463d-8ca4-8d289b64af1b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:08.047293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3aa9015e-78d8-463d-8ca4-8d289b64af1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:08.137820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:08.153188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 663/1342] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:08.192122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2953b7-bfe8-4b13-953f-1d73064159d1 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:08.194493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9a2953b7-bfe8-4b13-953f-1d73064159d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:08.289993 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2953b7-bfe8-4b13-953f-1d73064159d1 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:08.293491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c38d6-ad83-488d-b566-0c34d5f292c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:08.294132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c38d6-ad83-488d-b566-0c34d5f292c3 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:08.294970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c38d6-ad83-488d-b566-0c34d5f292c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:08.295253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c38d6-ad83-488d-b566-0c34d5f292c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:08.304402 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c38d6-ad83-488d-b566-0c34d5f292c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:08.305825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c38d6-ad83-488d-b566-0c34d5f292c3 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:08.315795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 680/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:11.117568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4bccd-8c59-4cbb-8fae-be6b87fb5fdc None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:11.118102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4bccd-8c59-4cbb-8fae-be6b87fb5fdc None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:11.118214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4bccd-8c59-4cbb-8fae-be6b87fb5fdc None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:11.119582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 664/1344] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:11.126168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacbc8f-c1fe-4683-8ab3-056f8aed1169 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:11.126625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacbc8f-c1fe-4683-8ab3-056f8aed1169 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:11.126880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacbc8f-c1fe-4683-8ab3-056f8aed1169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:11.160957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-8dacbc8f-c1fe-4683-8ab3-056f8aed1169 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:11.162334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8dacbc8f-c1fe-4683-8ab3-056f8aed1169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:11.194395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:11.243537 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 681/1345] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:11.280484 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4bccd-8c59-4cbb-8fae-be6b87fb5fdc None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:11.291044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4bccd-8c59-4cbb-8fae-be6b87fb5fdc None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:11.295127 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee63a4-c5ef-47ff-a208-d36f772f5a0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:11.295847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee63a4-c5ef-47ff-a208-d36f772f5a0f service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:11.296197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee63a4-c5ef-47ff-a208-d36f772f5a0f service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:11.296541 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee63a4-c5ef-47ff-a208-d36f772f5a0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:11.305508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee63a4-c5ef-47ff-a208-d36f772f5a0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:11.307010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee63a4-c5ef-47ff-a208-d36f772f5a0f service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:11.317042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 665/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:15.626497 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30c429-97f0-4bba-9222-65504a6579c0 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:15.627006 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30c429-97f0-4bba-9222-65504a6579c0 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:15.627118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30c429-97f0-4bba-9222-65504a6579c0 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:15.628452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 682/1347] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:15.633752 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5389d-1725-460a-a753-5d12aac963b8 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:15.634161 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5389d-1725-460a-a753-5d12aac963b8 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:15.634303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5389d-1725-460a-a753-5d12aac963b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:15.651737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-0fc5389d-1725-460a-a753-5d12aac963b8 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:15.652848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-0fc5389d-1725-460a-a753-5d12aac963b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:15.661297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 666/1348] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:15.699945 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30c429-97f0-4bba-9222-65504a6579c0 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:15.702336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2f30c429-97f0-4bba-9222-65504a6579c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:15.715136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30c429-97f0-4bba-9222-65504a6579c0 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:15.719515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b94040-116a-4dfb-8660-9d3f4f0d6f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:15.720241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b94040-116a-4dfb-8660-9d3f4f0d6f9e service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:15.720524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b94040-116a-4dfb-8660-9d3f4f0d6f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:15.720800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b94040-116a-4dfb-8660-9d3f4f0d6f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:15.730111 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b94040-116a-4dfb-8660-9d3f4f0d6f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:15.731553 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b94040-116a-4dfb-8660-9d3f4f0d6f9e service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:15.741173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 683/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:21.218372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a80163-84a4-43b9-ba80-0a8096c895e4 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:21.219185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a80163-84a4-43b9-ba80-0a8096c895e4 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:21.219640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a80163-84a4-43b9-ba80-0a8096c895e4 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:21.221252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 667/1350] 10.222.0.52 () {54 vars in 921 bytes} [Thu May 19 13:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:21.227092 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a6d9c-a011-4600-b416-3f671f2b49b7 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:21.227851 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a6d9c-a011-4600-b416-3f671f2b49b7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:21.228280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a6d9c-a011-4600-b416-3f671f2b49b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:21.304387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ba2a6d9c-a011-4600-b416-3f671f2b49b7 None None] MFA Rules not processed for user `b105f8163ce84a9ba725e6b04d348446`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:21.305824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ba2a6d9c-a011-4600-b416-3f671f2b49b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:21.411757 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 684/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Thu May 19 13:46:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:25.646521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=13 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:25.780354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb1d1b-289f-461e-a042-211ec9fe979a None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:25.781336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb1d1b-289f-461e-a042-211ec9fe979a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:25.782009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb1d1b-289f-461e-a042-211ec9fe979a None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:25.784604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 668/1352] 10.222.0.52 () {54 vars in 921 bytes} [Thu May 19 13:46:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:25.789879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db4b76-ba5c-4a76-905f-b52eac15e370 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:25.790979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db4b76-ba5c-4a76-905f-b52eac15e370 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:25.791718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db4b76-ba5c-4a76-905f-b52eac15e370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:25.816775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f1db4b76-ba5c-4a76-905f-b52eac15e370 None None] MFA Rules not processed for user `b105f8163ce84a9ba725e6b04d348446`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:25.817996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f1db4b76-ba5c-4a76-905f-b52eac15e370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:25.827361 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 685/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Thu May 19 13:46:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:27.263513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789cbbe-3dfe-4115-bf97-7ac573146cfd None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:27.264103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789cbbe-3dfe-4115-bf97-7ac573146cfd None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:27.264103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789cbbe-3dfe-4115-bf97-7ac573146cfd None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:27.265488 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 669/1354] 10.222.0.52 () {54 vars in 921 bytes} [Thu May 19 13:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:27.270631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7d2dd-acb1-4faf-aec5-e93b36e7855b None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:27.271104 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7d2dd-acb1-4faf-aec5-e93b36e7855b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:27.271201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7d2dd-acb1-4faf-aec5-e93b36e7855b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:27.290132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e2a7d2dd-acb1-4faf-aec5-e93b36e7855b None None] MFA Rules not processed for user `b105f8163ce84a9ba725e6b04d348446`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:27.291369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e2a7d2dd-acb1-4faf-aec5-e93b36e7855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:27.300109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 686/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Thu May 19 13:46:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:31.153428 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76b315-ccbb-43c7-9e07-ab5120804945 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:31.154023 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76b315-ccbb-43c7-9e07-ab5120804945 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:31.154023 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76b315-ccbb-43c7-9e07-ab5120804945 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:31.155349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 670/1356] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:31.160547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bff4f-f83e-49cb-b3db-dad6f68453ec None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:31.160964 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bff4f-f83e-49cb-b3db-dad6f68453ec None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:31.161118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bff4f-f83e-49cb-b3db-dad6f68453ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:31.178809 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-791bff4f-f83e-49cb-b3db-dad6f68453ec None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:31.180029 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-791bff4f-f83e-49cb-b3db-dad6f68453ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:31.188860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 687/1357] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:31.241660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0a3ec-0362-4647-825a-f145172cd27b None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:31.242148 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0a3ec-0362-4647-825a-f145172cd27b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:31.242831 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0a3ec-0362-4647-825a-f145172cd27b None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:31.244631 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 671/1358] 10.222.0.52 () {54 vars in 961 bytes} [Thu May 19 13:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:31.252806 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00bf48-a636-4a82-bbf4-dde10dfa879f None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:31.253096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00bf48-a636-4a82-bbf4-dde10dfa879f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:31.253378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00bf48-a636-4a82-bbf4-dde10dfa879f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:31.302174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:31.311417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-8b00bf48-a636-4a82-bbf4-dde10dfa879f None None] MFA Rules not processed for user `4f6093d0c1fc41d98c58bace81e3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:31.312616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8b00bf48-a636-4a82-bbf4-dde10dfa879f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:31.431075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 688/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Thu May 19 13:46:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:31.432317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0a3ec-0362-4647-825a-f145172cd27b None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:31.449963 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0a3ec-0362-4647-825a-f145172cd27b None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:31.456928 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5534df4-9f7a-4ee6-ac1e-95359e54045f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:31.457993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5534df4-9f7a-4ee6-ac1e-95359e54045f service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:31.458411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5534df4-9f7a-4ee6-ac1e-95359e54045f service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:31.458748 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5534df4-9f7a-4ee6-ac1e-95359e54045f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:31.471539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5534df4-9f7a-4ee6-ac1e-95359e54045f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:31.473279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5534df4-9f7a-4ee6-ac1e-95359e54045f service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:31.485776 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 672/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:46:33.949605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6414a-3012-4904-a6ef-687f830ac00f None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:33.950120 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6414a-3012-4904-a6ef-687f830ac00f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:33.950397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6414a-3012-4904-a6ef-687f830ac00f None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:33.951821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 689/1361] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:33.956952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ec86d-2315-4d18-b830-204c8a64bd14 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:33.957350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ec86d-2315-4d18-b830-204c8a64bd14 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:33.957510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ec86d-2315-4d18-b830-204c8a64bd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:33.975832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-ab4ec86d-2315-4d18-b830-204c8a64bd14 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:33.977039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ab4ec86d-2315-4d18-b830-204c8a64bd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:33.985639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 673/1362] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:34.048275 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6414a-3012-4904-a6ef-687f830ac00f None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:34.066234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6414a-3012-4904-a6ef-687f830ac00f None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:34.071834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59abf033-dfaa-4025-b213-4b917c4c6017 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:34.072554 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abf033-dfaa-4025-b213-4b917c4c6017 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:34.072835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abf033-dfaa-4025-b213-4b917c4c6017 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:34.073248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abf033-dfaa-4025-b213-4b917c4c6017 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:34.082709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abf033-dfaa-4025-b213-4b917c4c6017 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:34.084352 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abf033-dfaa-4025-b213-4b917c4c6017 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:34.094228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 690/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:36.671810 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06119a-9c7d-4f9f-8cdd-fec2860223a1 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:36.672273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06119a-9c7d-4f9f-8cdd-fec2860223a1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:36.672372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06119a-9c7d-4f9f-8cdd-fec2860223a1 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:36.673720 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 674/1364] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:36.679114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854d5ab-5614-4433-b835-cc93d8f5be2a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:36.679542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854d5ab-5614-4433-b835-cc93d8f5be2a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:36.679681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854d5ab-5614-4433-b835-cc93d8f5be2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:36.697731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-2854d5ab-5614-4433-b835-cc93d8f5be2a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:36.698948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2854d5ab-5614-4433-b835-cc93d8f5be2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:36.709971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 691/1365] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:36.759719 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13747e-c0c6-4945-b041-4dbec78e3214 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:36.760521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13747e-c0c6-4945-b041-4dbec78e3214 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:36.761041 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13747e-c0c6-4945-b041-4dbec78e3214 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:36.763409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 675/1366] 10.222.0.52 () {54 vars in 961 bytes} [Thu May 19 13:46:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:36.781122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec960ea-125c-45a0-b800-95f883feabe2 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:36.781736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec960ea-125c-45a0-b800-95f883feabe2 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:36.782266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec960ea-125c-45a0-b800-95f883feabe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:36.804514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-0ec960ea-125c-45a0-b800-95f883feabe2 None None] MFA Rules not processed for user `4f6093d0c1fc41d98c58bace81e3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:36.805622 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-0ec960ea-125c-45a0-b800-95f883feabe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:36.815726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 692/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Thu May 19 13:46:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:36.824359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13747e-c0c6-4945-b041-4dbec78e3214 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:36.838616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13747e-c0c6-4945-b041-4dbec78e3214 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:36.844241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c831208-646b-4bc0-b3a8-ac4cdeffab6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:36.844874 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c831208-646b-4bc0-b3a8-ac4cdeffab6b service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:36.845155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c831208-646b-4bc0-b3a8-ac4cdeffab6b service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:36.845430 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c831208-646b-4bc0-b3a8-ac4cdeffab6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:36.853666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c831208-646b-4bc0-b3a8-ac4cdeffab6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:36.855257 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c831208-646b-4bc0-b3a8-ac4cdeffab6b service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:36.865402 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 676/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:39.154931 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d181ec-5953-475e-9a9e-f3078cf57926 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:39.155512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d181ec-5953-475e-9a9e-f3078cf57926 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:39.155512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d181ec-5953-475e-9a9e-f3078cf57926 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:39.157048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 693/1369] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:39.162308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb801033-35b3-4a6c-8157-2d67bf2e999e None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:39.162731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb801033-35b3-4a6c-8157-2d67bf2e999e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:39.162899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb801033-35b3-4a6c-8157-2d67bf2e999e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:39.181347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-fb801033-35b3-4a6c-8157-2d67bf2e999e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:39.182833 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-fb801033-35b3-4a6c-8157-2d67bf2e999e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:39.192020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 677/1370] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:39.235357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d181ec-5953-475e-9a9e-f3078cf57926 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:39.252509 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d181ec-5953-475e-9a9e-f3078cf57926 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:39.258343 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb793b0-d1bb-4d0f-ac9a-d3da3258b3c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:39.259029 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb793b0-d1bb-4d0f-ac9a-d3da3258b3c2 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:39.259321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb793b0-d1bb-4d0f-ac9a-d3da3258b3c2 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:39.259604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb793b0-d1bb-4d0f-ac9a-d3da3258b3c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:39.268894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb793b0-d1bb-4d0f-ac9a-d3da3258b3c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:39.270478 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb793b0-d1bb-4d0f-ac9a-d3da3258b3c2 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:39.281707 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 694/1371] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:39.511383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2395923-b1fe-41ce-a31d-bc139ff73640 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:39.511588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2395923-b1fe-41ce-a31d-bc139ff73640 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:39.511857 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2395923-b1fe-41ce-a31d-bc139ff73640 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:39.513495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 678/1372] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:39.519574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c52e22-bac4-4fe7-a8be-cb54ebaa2eb4 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:39.519870 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c52e22-bac4-4fe7-a8be-cb54ebaa2eb4 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:39.521507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c52e22-bac4-4fe7-a8be-cb54ebaa2eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:39.541284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-30c52e22-bac4-4fe7-a8be-cb54ebaa2eb4 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:39.542516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-30c52e22-bac4-4fe7-a8be-cb54ebaa2eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:39.552560 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 695/1373] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:39.865318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750989e-770b-48f1-848b-ae0286dca854 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:39.865677 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750989e-770b-48f1-848b-ae0286dca854 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:39.866031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750989e-770b-48f1-848b-ae0286dca854 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:39.867509 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 679/1374] 10.222.0.52 () {54 vars in 963 bytes} [Thu May 19 13:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:39.873858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63107b4-6dd1-408c-a9a5-99e4a5e28a60 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:39.874555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63107b4-6dd1-408c-a9a5-99e4a5e28a60 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:39.874930 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63107b4-6dd1-408c-a9a5-99e4a5e28a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:39.926046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e63107b4-6dd1-408c-a9a5-99e4a5e28a60 None None] MFA Rules not processed for user `1ed0973226a2446ebf7c0cc3ee04e446`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:39.927433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e63107b4-6dd1-408c-a9a5-99e4a5e28a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:39.991787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 696/1375] 10.222.0.52 () {58 vars in 1060 bytes} [Thu May 19 13:46:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:40.000037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750989e-770b-48f1-848b-ae0286dca854 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:40.011207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750989e-770b-48f1-848b-ae0286dca854 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:40.015127 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c78b2-e0f1-4229-bf61-910518d180ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:40.015994 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c78b2-e0f1-4229-bf61-910518d180ed service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:40.016443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c78b2-e0f1-4229-bf61-910518d180ed service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:40.016880 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c78b2-e0f1-4229-bf61-910518d180ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:40.036529 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c78b2-e0f1-4229-bf61-910518d180ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:40.038411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c78b2-e0f1-4229-bf61-910518d180ed service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:40.061217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 680/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:41.556461 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14646c35-9433-42c0-bd3f-9451440201d4 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:41.557022 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14646c35-9433-42c0-bd3f-9451440201d4 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:41.557022 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14646c35-9433-42c0-bd3f-9451440201d4 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:41.558470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 697/1377] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:41.564446 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d398eb6-2a23-40de-94ec-79e302e05d45 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:41.564910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d398eb6-2a23-40de-94ec-79e302e05d45 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:41.565026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d398eb6-2a23-40de-94ec-79e302e05d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:41.583781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-2d398eb6-2a23-40de-94ec-79e302e05d45 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:41.584982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2d398eb6-2a23-40de-94ec-79e302e05d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:41.592928 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 681/1378] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:41.636384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14646c35-9433-42c0-bd3f-9451440201d4 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:41.650974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14646c35-9433-42c0-bd3f-9451440201d4 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:41.656531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2416e4-4858-4281-b768-8732b92d59db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:41.657166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2416e4-4858-4281-b768-8732b92d59db service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:41.657451 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2416e4-4858-4281-b768-8732b92d59db service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:41.657731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2416e4-4858-4281-b768-8732b92d59db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:41.665985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2416e4-4858-4281-b768-8732b92d59db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:41.667634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2416e4-4858-4281-b768-8732b92d59db service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:41.671960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:41.679980 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 698/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:43.978641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ba043-c0ab-4585-9bcc-4bc2dad66efc None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:43.979621 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ba043-c0ab-4585-9bcc-4bc2dad66efc None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:43.980095 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ba043-c0ab-4585-9bcc-4bc2dad66efc None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:43.981767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 682/1380] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:43.988588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dab08e-09d4-426b-a390-c23b22bb7ceb None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:43.989386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dab08e-09d4-426b-a390-c23b22bb7ceb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:43.989841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dab08e-09d4-426b-a390-c23b22bb7ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:44.011666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e3dab08e-09d4-426b-a390-c23b22bb7ceb None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:44.013772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e3dab08e-09d4-426b-a390-c23b22bb7ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:44.028405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 699/1381] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:44.081345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ba043-c0ab-4585-9bcc-4bc2dad66efc None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:44.096807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ba043-c0ab-4585-9bcc-4bc2dad66efc None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:44.102264 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9030664-f4ca-4453-bd1b-df006f0275e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:44.102925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9030664-f4ca-4453-bd1b-df006f0275e9 service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:44.103210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9030664-f4ca-4453-bd1b-df006f0275e9 service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:44.103489 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9030664-f4ca-4453-bd1b-df006f0275e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:44.118434 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9030664-f4ca-4453-bd1b-df006f0275e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:44.120050 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9030664-f4ca-4453-bd1b-df006f0275e9 service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:44.130560 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 683/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:46.404860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e090c7-b125-46e2-946b-0484e534e93a None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:46.405339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e090c7-b125-46e2-946b-0484e534e93a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:46.405425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e090c7-b125-46e2-946b-0484e534e93a None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:46.406785 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 700/1383] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:46.412088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210f652-e336-416c-98e1-05f6a8521509 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:46.412505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210f652-e336-416c-98e1-05f6a8521509 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:46.412653 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210f652-e336-416c-98e1-05f6a8521509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:46.431882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a210f652-e336-416c-98e1-05f6a8521509 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:46.433282 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a210f652-e336-416c-98e1-05f6a8521509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:46.442527 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 684/1384] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:46.485811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e090c7-b125-46e2-946b-0484e534e93a None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:46.501583 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e090c7-b125-46e2-946b-0484e534e93a None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:46.507356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d3bf5-93e0-4324-a04a-e6e48d1bc433 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:46.508077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d3bf5-93e0-4324-a04a-e6e48d1bc433 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:46.508433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d3bf5-93e0-4324-a04a-e6e48d1bc433 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:46.508793 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d3bf5-93e0-4324-a04a-e6e48d1bc433 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:46.518637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d3bf5-93e0-4324-a04a-e6e48d1bc433 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:46.520368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d3bf5-93e0-4324-a04a-e6e48d1bc433 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:46.535041 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 701/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:48.759859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0ebfc-02a2-4635-a4b4-6dbbd7f1ecb8 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:48.760371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0ebfc-02a2-4635-a4b4-6dbbd7f1ecb8 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:48.760371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0ebfc-02a2-4635-a4b4-6dbbd7f1ecb8 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:48.761735 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 685/1386] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:48.767765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f596e-9bc0-49de-bf62-f5af1c9aa3ea None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:48.768225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f596e-9bc0-49de-bf62-f5af1c9aa3ea None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:48.768349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f596e-9bc0-49de-bf62-f5af1c9aa3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:48.786554 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-4d7f596e-9bc0-49de-bf62-f5af1c9aa3ea None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:48.787826 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4d7f596e-9bc0-49de-bf62-f5af1c9aa3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:48.796643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 702/1387] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:48.842318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0ebfc-02a2-4635-a4b4-6dbbd7f1ecb8 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:48.858510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0ebfc-02a2-4635-a4b4-6dbbd7f1ecb8 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:48.864037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67446213-d7bf-474b-8b8a-42c4dc279b85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:48.864692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67446213-d7bf-474b-8b8a-42c4dc279b85 service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:48.864993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67446213-d7bf-474b-8b8a-42c4dc279b85 service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:48.865280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67446213-d7bf-474b-8b8a-42c4dc279b85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:48.874647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67446213-d7bf-474b-8b8a-42c4dc279b85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:48.876300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67446213-d7bf-474b-8b8a-42c4dc279b85 service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:48.885909 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 686/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:51.166522 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcbc5f-b7e2-4b31-937e-bf1c5ec0feec None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:51.167099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcbc5f-b7e2-4b31-937e-bf1c5ec0feec None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:51.167099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcbc5f-b7e2-4b31-937e-bf1c5ec0feec None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:51.168987 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 703/1389] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:51.175114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea3100-1f4e-4c3c-ab87-9cc968c37f5b None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:51.175949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea3100-1f4e-4c3c-ab87-9cc968c37f5b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:51.176244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea3100-1f4e-4c3c-ab87-9cc968c37f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:51.195710 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-d8ea3100-1f4e-4c3c-ab87-9cc968c37f5b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:51.196940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d8ea3100-1f4e-4c3c-ab87-9cc968c37f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:51.205175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 687/1390] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:51.251062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcbc5f-b7e2-4b31-937e-bf1c5ec0feec None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:51.267417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcbc5f-b7e2-4b31-937e-bf1c5ec0feec None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:51.273242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60a1d6-1ebf-4056-8965-48914e68e1af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:51.273922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60a1d6-1ebf-4056-8965-48914e68e1af service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:51.274155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60a1d6-1ebf-4056-8965-48914e68e1af service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:51.274429 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60a1d6-1ebf-4056-8965-48914e68e1af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:51.283540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60a1d6-1ebf-4056-8965-48914e68e1af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:51.285107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60a1d6-1ebf-4056-8965-48914e68e1af service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:51.295198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 704/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:53.564661 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19db03-4b46-417a-a540-7af096685e86 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:53.565217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19db03-4b46-417a-a540-7af096685e86 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:53.565217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19db03-4b46-417a-a540-7af096685e86 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:53.566589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 688/1392] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:46:53.573102 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b254f0-f83f-414a-abfb-a3bdf3b61cc4 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:53.573500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b254f0-f83f-414a-abfb-a3bdf3b61cc4 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:53.573816 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b254f0-f83f-414a-abfb-a3bdf3b61cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:53.596677 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e2b254f0-f83f-414a-abfb-a3bdf3b61cc4 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:53.598282 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e2b254f0-f83f-414a-abfb-a3bdf3b61cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:53.609382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 705/1393] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:53.660492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19db03-4b46-417a-a540-7af096685e86 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:53.678804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19db03-4b46-417a-a540-7af096685e86 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:53.686201 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e28e95-decd-4f64-ac40-d019b11fd370 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:53.687033 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e28e95-decd-4f64-ac40-d019b11fd370 service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:53.687386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e28e95-decd-4f64-ac40-d019b11fd370 service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:53.687734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e28e95-decd-4f64-ac40-d019b11fd370 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:53.698725 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e28e95-decd-4f64-ac40-d019b11fd370 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:53.700895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e28e95-decd-4f64-ac40-d019b11fd370 service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:53.713846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 689/1394] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:56.015489 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6506d5-4e64-402c-9e7f-1ebc2359aecb None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:56.016099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6506d5-4e64-402c-9e7f-1ebc2359aecb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:56.016099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6506d5-4e64-402c-9e7f-1ebc2359aecb None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:56.017426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 706/1395] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:56.022715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe80230-62a1-4d4a-a7f2-2f1c90e2e381 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:56.023182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe80230-62a1-4d4a-a7f2-2f1c90e2e381 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:56.023336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe80230-62a1-4d4a-a7f2-2f1c90e2e381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:56.029609 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=52 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:56.042171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-1fe80230-62a1-4d4a-a7f2-2f1c90e2e381 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:56.043421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1fe80230-62a1-4d4a-a7f2-2f1c90e2e381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:56.051395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 690/1396] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:56.095762 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6506d5-4e64-402c-9e7f-1ebc2359aecb None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:56.110249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6506d5-4e64-402c-9e7f-1ebc2359aecb None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:56.115714 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d430f11-53b7-4f62-8d26-c5077aee6e8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:56.116348 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d430f11-53b7-4f62-8d26-c5077aee6e8b service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:56.116637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d430f11-53b7-4f62-8d26-c5077aee6e8b service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:56.116911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d430f11-53b7-4f62-8d26-c5077aee6e8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:56.125978 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d430f11-53b7-4f62-8d26-c5077aee6e8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:56.127586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d430f11-53b7-4f62-8d26-c5077aee6e8b service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:56.137955 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 707/1397] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:57.064122 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e42f6-a6fe-4797-a47e-53da1b5557c4 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:57.064657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e42f6-a6fe-4797-a47e-53da1b5557c4 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:57.064657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e42f6-a6fe-4797-a47e-53da1b5557c4 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:57.065991 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 691/1398] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:46:57.071498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75406897-3e1f-402f-9d29-d300be60c3bc None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:57.071908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75406897-3e1f-402f-9d29-d300be60c3bc None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:57.072068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75406897-3e1f-402f-9d29-d300be60c3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:57.089797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-75406897-3e1f-402f-9d29-d300be60c3bc None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:57.091126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-75406897-3e1f-402f-9d29-d300be60c3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:57.098945 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 708/1399] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:57.154752 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b40f19-e7e8-49ec-80b3-823224ce3f3a None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:57.155080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b40f19-e7e8-49ec-80b3-823224ce3f3a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:57.155356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b40f19-e7e8-49ec-80b3-823224ce3f3a None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:57.156704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 692/1400] 10.222.0.52 () {54 vars in 962 bytes} [Thu May 19 13:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:57.161946 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcfe5d-b6d1-406c-ae4d-36f4d640fbef None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:57.162241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcfe5d-b6d1-406c-ae4d-36f4d640fbef None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:57.162524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcfe5d-b6d1-406c-ae4d-36f4d640fbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:57.211724 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-5abcfe5d-b6d1-406c-ae4d-36f4d640fbef None None] MFA Rules not processed for user `666ade4ce1fe447487998d086779389b`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:57.212956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5abcfe5d-b6d1-406c-ae4d-36f4d640fbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:57.275405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 709/1401] 10.222.0.52 () {58 vars in 1059 bytes} [Thu May 19 13:46:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:57.283398 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b40f19-e7e8-49ec-80b3-823224ce3f3a None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:57.295217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b40f19-e7e8-49ec-80b3-823224ce3f3a None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:57.298973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c9e16-5594-423f-9df8-317bd62e3576 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '666ade4ce1fe447487998d086779389b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:57.299746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c9e16-5594-423f-9df8-317bd62e3576 service glance] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:57.300100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c9e16-5594-423f-9df8-317bd62e3576 service glance] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:57.300483 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c9e16-5594-423f-9df8-317bd62e3576 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:57.310288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c9e16-5594-423f-9df8-317bd62e3576 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:57.311743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c9e16-5594-423f-9df8-317bd62e3576 service glance] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:57.321238 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 693/1402] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:57.882352 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9a1d4-57c4-4f42-9cab-90545f3376ea None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:57.882652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9a1d4-57c4-4f42-9cab-90545f3376ea None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:57.883036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9a1d4-57c4-4f42-9cab-90545f3376ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:57.932695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-2df9a1d4-57c4-4f42-9cab-90545f3376ea None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:57.934661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2df9a1d4-57c4-4f42-9cab-90545f3376ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:57.976252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:58.002280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 710/1403] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 13:46:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:58.047318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899227f-e067-46dd-8629-3743c99ecc06 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:58.047612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899227f-e067-46dd-8629-3743c99ecc06 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:58.047892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899227f-e067-46dd-8629-3743c99ecc06 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:58.049267 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 694/1404] 10.222.0.52 () {54 vars in 942 bytes} [Thu May 19 13:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:58.055764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d468e-8222-452d-b5d3-43be5eeb8cdd None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:58.056058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d468e-8222-452d-b5d3-43be5eeb8cdd None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:58.056340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d468e-8222-452d-b5d3-43be5eeb8cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:58.105183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-d68d468e-8222-452d-b5d3-43be5eeb8cdd None None] MFA Rules not processed for user `8554ff40bbcb4fbb801d854d6527a6d2`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:58.107040 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d68d468e-8222-452d-b5d3-43be5eeb8cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:58.181444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 711/1405] 10.222.0.52 () {58 vars in 1039 bytes} [Thu May 19 13:46:58 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:58.191061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899227f-e067-46dd-8629-3743c99ecc06 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:58.204873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899227f-e067-46dd-8629-3743c99ecc06 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:58.209862 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b8d23-df67-4edc-bac2-85c2270c0594 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:58.210508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b8d23-df67-4edc-bac2-85c2270c0594 service swift] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:58.210796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b8d23-df67-4edc-bac2-85c2270c0594 service swift] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:58.211104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b8d23-df67-4edc-bac2-85c2270c0594 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:58.221092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b8d23-df67-4edc-bac2-85c2270c0594 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:58.222712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b8d23-df67-4edc-bac2-85c2270c0594 service swift] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:58.233426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 695/1406] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 13:46:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:46:58.415285 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada49e0-0742-4e41-9cd2-fb80115d196c None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:58.415569 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada49e0-0742-4e41-9cd2-fb80115d196c None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:58.415848 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada49e0-0742-4e41-9cd2-fb80115d196c None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:58.417496 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 712/1407] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:46:58.423340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816ae32-53a5-49e0-8bd8-0a27a332ff7f None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:58.423858 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816ae32-53a5-49e0-8bd8-0a27a332ff7f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:58.424140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816ae32-53a5-49e0-8bd8-0a27a332ff7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:58.445256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-e816ae32-53a5-49e0-8bd8-0a27a332ff7f None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:58.446730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e816ae32-53a5-49e0-8bd8-0a27a332ff7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:58.456996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 696/1408] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:58.502831 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada49e0-0742-4e41-9cd2-fb80115d196c None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:58.521778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada49e0-0742-4e41-9cd2-fb80115d196c None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:58.527381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa7ac3-47fe-4d8b-8a25-b2d62a60ce1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:58.528860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa7ac3-47fe-4d8b-8a25-b2d62a60ce1f service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:58.529220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa7ac3-47fe-4d8b-8a25-b2d62a60ce1f service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:58.529564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa7ac3-47fe-4d8b-8a25-b2d62a60ce1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:58.541313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa7ac3-47fe-4d8b-8a25-b2d62a60ce1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:58.542981 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa7ac3-47fe-4d8b-8a25-b2d62a60ce1f service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:58.553848 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 713/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:46:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:08.437585 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=17 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:47:08.549142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=59 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:47:22.686562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed9b40-734b-4c3c-afd9-47f7942dad7e None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:22.687201 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed9b40-734b-4c3c-afd9-47f7942dad7e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:22.687201 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed9b40-734b-4c3c-afd9-47f7942dad7e None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:22.688412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 697/1410] 10.222.0.52 () {54 vars in 918 bytes} [Thu May 19 13:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:22.693645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c986fa-3bf3-432a-b9e2-9440c8157dea None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:22.694138 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c986fa-3bf3-432a-b9e2-9440c8157dea None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:22.694254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c986fa-3bf3-432a-b9e2-9440c8157dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:22.712439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-c4c986fa-3bf3-432a-b9e2-9440c8157dea None None] MFA Rules not processed for user `b105f8163ce84a9ba725e6b04d348446`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:22.713663 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c4c986fa-3bf3-432a-b9e2-9440c8157dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:22.722482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 714/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Thu May 19 13:47:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:22.737164 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f77dc8-eb67-4bd0-af45-ad74e1b207dd None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:22.737456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f77dc8-eb67-4bd0-af45-ad74e1b207dd None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:22.737738 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f77dc8-eb67-4bd0-af45-ad74e1b207dd None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:22.739122 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 698/1412] 10.222.0.52 () {54 vars in 918 bytes} [Thu May 19 13:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:22.743959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ba49c-f4a9-4b7a-bd20-071da346c148 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:22.744450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ba49c-f4a9-4b7a-bd20-071da346c148 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:22.744736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ba49c-f4a9-4b7a-bd20-071da346c148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:22.763691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-904ba49c-f4a9-4b7a-bd20-071da346c148 None None] MFA Rules not processed for user `4f6093d0c1fc41d98c58bace81e3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:22.764898 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-904ba49c-f4a9-4b7a-bd20-071da346c148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:22.773954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 715/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Thu May 19 13:47:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:25.457274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8d087-8ee0-4f2e-88e1-bc2fc115995e None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:25.457824 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8d087-8ee0-4f2e-88e1-bc2fc115995e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:25.457824 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8d087-8ee0-4f2e-88e1-bc2fc115995e None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:25.459239 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:25.464468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c45a0-35ab-4a46-877c-272bbffcc664 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:25.465713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c45a0-35ab-4a46-877c-272bbffcc664 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:25.465853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c45a0-35ab-4a46-877c-272bbffcc664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:25.483132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ce0c45a0-35ab-4a46-877c-272bbffcc664 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:25.484324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ce0c45a0-35ab-4a46-877c-272bbffcc664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:25.492080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:25.536025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8d087-8ee0-4f2e-88e1-bc2fc115995e None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:25.550768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8d087-8ee0-4f2e-88e1-bc2fc115995e None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:25.556081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0fc9e-1d3c-4065-a65d-e17b5183a5ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:25.556709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0fc9e-1d3c-4065-a65d-e17b5183a5ff service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:25.556987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0fc9e-1d3c-4065-a65d-e17b5183a5ff service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:25.557263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0fc9e-1d3c-4065-a65d-e17b5183a5ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:25.565476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0fc9e-1d3c-4065-a65d-e17b5183a5ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:25.567084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0fc9e-1d3c-4065-a65d-e17b5183a5ff service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:25.576893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 700/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:26.571787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372dd77f-9b52-4bec-bfd6-340722af72a7 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:26.572302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372dd77f-9b52-4bec-bfd6-340722af72a7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:26.572410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372dd77f-9b52-4bec-bfd6-340722af72a7 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:26.573742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:26.579261 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bac55e-b9f7-4522-949a-7065ec27f396 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:26.579697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bac55e-b9f7-4522-949a-7065ec27f396 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:26.579808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bac55e-b9f7-4522-949a-7065ec27f396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:26.597670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-36bac55e-b9f7-4522-949a-7065ec27f396 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:26.598853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-36bac55e-b9f7-4522-949a-7065ec27f396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:26.606782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:26.813981 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372dd77f-9b52-4bec-bfd6-340722af72a7 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:26.828548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372dd77f-9b52-4bec-bfd6-340722af72a7 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:26.833864 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a163f-a59a-4787-b155-d7639c25a875 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:26.834489 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a163f-a59a-4787-b155-d7639c25a875 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:26.834773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a163f-a59a-4787-b155-d7639c25a875 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:26.835083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a163f-a59a-4787-b155-d7639c25a875 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:26.843082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a163f-a59a-4787-b155-d7639c25a875 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:26.844918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a163f-a59a-4787-b155-d7639c25a875 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:26.854134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 718/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:27.801042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd78da-ec6a-472d-b037-a8b488674c0f None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:27.801582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd78da-ec6a-472d-b037-a8b488674c0f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:27.801582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd78da-ec6a-472d-b037-a8b488674c0f None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:27.802890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:27.807845 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e89095-2b0a-48e5-acf3-aa7c4a9df1f8 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:27.808321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e89095-2b0a-48e5-acf3-aa7c4a9df1f8 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:27.808441 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e89095-2b0a-48e5-acf3-aa7c4a9df1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:27.826712 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f5e89095-2b0a-48e5-acf3-aa7c4a9df1f8 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:27.827918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f5e89095-2b0a-48e5-acf3-aa7c4a9df1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:27.835949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:27.984468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd78da-ec6a-472d-b037-a8b488674c0f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:27.998605 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd78da-ec6a-472d-b037-a8b488674c0f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:28.004390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19697c43-1ce7-4cf6-844e-9fe31d79bc7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:28.005075 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19697c43-1ce7-4cf6-844e-9fe31d79bc7f service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:28.005358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19697c43-1ce7-4cf6-844e-9fe31d79bc7f service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:28.005632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19697c43-1ce7-4cf6-844e-9fe31d79bc7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:28.014502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19697c43-1ce7-4cf6-844e-9fe31d79bc7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:28.016132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19697c43-1ce7-4cf6-844e-9fe31d79bc7f service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:28.025312 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 703/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:28.950344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9023822-7f75-4590-b9d5-e79dcd22bcd2 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:28.950906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9023822-7f75-4590-b9d5-e79dcd22bcd2 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:28.951030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9023822-7f75-4590-b9d5-e79dcd22bcd2 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:28.952309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:28.957042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88284f-3b26-445e-9689-823513d50fb1 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:28.957458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88284f-3b26-445e-9689-823513d50fb1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:28.957594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88284f-3b26-445e-9689-823513d50fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:28.975624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6a88284f-3b26-445e-9689-823513d50fb1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:28.976801 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6a88284f-3b26-445e-9689-823513d50fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:28.985062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:29.133420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9023822-7f75-4590-b9d5-e79dcd22bcd2 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:29.148843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9023822-7f75-4590-b9d5-e79dcd22bcd2 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:29.154955 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cbb89-609f-4faf-8e43-a8b5f57ec226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:29.155623 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cbb89-609f-4faf-8e43-a8b5f57ec226 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:29.155909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cbb89-609f-4faf-8e43-a8b5f57ec226 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:29.156189 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cbb89-609f-4faf-8e43-a8b5f57ec226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:29.164411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cbb89-609f-4faf-8e43-a8b5f57ec226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:29.165997 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cbb89-609f-4faf-8e43-a8b5f57ec226 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:29.176325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 721/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:30.098716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79c047-edbb-493f-9ce7-89e50c98c8bf None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:30.099286 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79c047-edbb-493f-9ce7-89e50c98c8bf None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:30.099286 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79c047-edbb-493f-9ce7-89e50c98c8bf None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:30.100660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:30.105477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766c3a8-a885-4e06-b3fd-f4130b16e6b6 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:30.105888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766c3a8-a885-4e06-b3fd-f4130b16e6b6 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:30.106034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766c3a8-a885-4e06-b3fd-f4130b16e6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:30.123493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-d766c3a8-a885-4e06-b3fd-f4130b16e6b6 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:30.124940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d766c3a8-a885-4e06-b3fd-f4130b16e6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:30.132548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:30.282022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79c047-edbb-493f-9ce7-89e50c98c8bf None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:30.296419 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79c047-edbb-493f-9ce7-89e50c98c8bf None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:30.301698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a72d53-a222-4d35-9c33-9487a7c2ae6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:30.302332 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a72d53-a222-4d35-9c33-9487a7c2ae6f service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:30.302616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a72d53-a222-4d35-9c33-9487a7c2ae6f service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:30.302916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a72d53-a222-4d35-9c33-9487a7c2ae6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:30.311500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a72d53-a222-4d35-9c33-9487a7c2ae6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:30.313063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a72d53-a222-4d35-9c33-9487a7c2ae6f service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:30.322233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 706/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:31.204140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce18e29-ae3b-42e4-a64d-eca9e967ed8e None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:31.204683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce18e29-ae3b-42e4-a64d-eca9e967ed8e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:31.204776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce18e29-ae3b-42e4-a64d-eca9e967ed8e None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:31.206119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:31.210966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eb8a9-6624-4519-9fe0-4fdfbeef8fe9 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:31.211393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eb8a9-6624-4519-9fe0-4fdfbeef8fe9 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:31.211513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eb8a9-6624-4519-9fe0-4fdfbeef8fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:31.229867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-8e0eb8a9-6624-4519-9fe0-4fdfbeef8fe9 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:31.231677 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-8e0eb8a9-6624-4519-9fe0-4fdfbeef8fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:31.239435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:31.389992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce18e29-ae3b-42e4-a64d-eca9e967ed8e None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:31.406508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce18e29-ae3b-42e4-a64d-eca9e967ed8e None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:31.411869 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ab02a-3ced-451e-9132-955fdf507bf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:31.412496 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ab02a-3ced-451e-9132-955fdf507bf9 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:31.412782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ab02a-3ced-451e-9132-955fdf507bf9 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:31.413054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ab02a-3ced-451e-9132-955fdf507bf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:31.421831 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ab02a-3ced-451e-9132-955fdf507bf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:31.423432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ab02a-3ced-451e-9132-955fdf507bf9 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:31.432996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 724/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:32.298609 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb279b3-88fd-4430-8137-2a50d5ba3dd7 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:32.299206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb279b3-88fd-4430-8137-2a50d5ba3dd7 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:32.299206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb279b3-88fd-4430-8137-2a50d5ba3dd7 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:32.300606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:32.305610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9666b-0b55-446b-a0be-ad68ce8eca48 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:32.306050 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9666b-0b55-446b-a0be-ad68ce8eca48 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:32.306147 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9666b-0b55-446b-a0be-ad68ce8eca48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:32.324182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-87f9666b-0b55-446b-a0be-ad68ce8eca48 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:32.325365 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-87f9666b-0b55-446b-a0be-ad68ce8eca48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:32.332991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:32.483508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb279b3-88fd-4430-8137-2a50d5ba3dd7 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:32.497709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb279b3-88fd-4430-8137-2a50d5ba3dd7 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:32.503368 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377f078-d18d-4228-8356-15db853f8ee6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:32.504000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377f078-d18d-4228-8356-15db853f8ee6 service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:32.504280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377f078-d18d-4228-8356-15db853f8ee6 service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:32.504555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377f078-d18d-4228-8356-15db853f8ee6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:32.513442 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377f078-d18d-4228-8356-15db853f8ee6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:32.515037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377f078-d18d-4228-8356-15db853f8ee6 service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:32.524497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 709/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:33.400235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29046f-802f-4c49-a3e7-5dc4d0d490bc None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:33.400235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29046f-802f-4c49-a3e7-5dc4d0d490bc None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:33.400235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29046f-802f-4c49-a3e7-5dc4d0d490bc None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:33.401508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:47:33.406723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102182-c96e-4a0a-9641-bfc887ce5de1 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:33.407203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102182-c96e-4a0a-9641-bfc887ce5de1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:33.407328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102182-c96e-4a0a-9641-bfc887ce5de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:33.425194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a3102182-c96e-4a0a-9641-bfc887ce5de1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:33.426304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a3102182-c96e-4a0a-9641-bfc887ce5de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:33.435010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:47:33.585731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29046f-802f-4c49-a3e7-5dc4d0d490bc None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:33.600460 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29046f-802f-4c49-a3e7-5dc4d0d490bc None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:33.606066 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc49878-701a-4c74-ac2c-7aabf04998d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:33.606695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc49878-701a-4c74-ac2c-7aabf04998d5 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:33.607018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc49878-701a-4c74-ac2c-7aabf04998d5 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:33.607295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc49878-701a-4c74-ac2c-7aabf04998d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:33.615448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc49878-701a-4c74-ac2c-7aabf04998d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:33.617019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc49878-701a-4c74-ac2c-7aabf04998d5 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:33.627175 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 727/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:34.509271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efb2b8-0626-49d4-b1ad-a92a3a4753bc None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:34.509807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efb2b8-0626-49d4-b1ad-a92a3a4753bc None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:34.509807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efb2b8-0626-49d4-b1ad-a92a3a4753bc None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:34.511270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:34.516224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1672cc-7277-4553-929f-07826f9f9aed None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:34.516614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1672cc-7277-4553-929f-07826f9f9aed None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:34.516787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1672cc-7277-4553-929f-07826f9f9aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:34.534891 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-2b1672cc-7277-4553-929f-07826f9f9aed None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:34.536099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2b1672cc-7277-4553-929f-07826f9f9aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:34.544388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:34.691648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efb2b8-0626-49d4-b1ad-a92a3a4753bc None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:34.706661 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efb2b8-0626-49d4-b1ad-a92a3a4753bc None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:34.712214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9391b3d-dfbb-40ec-b79b-189ba6e3cba6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:34.712847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9391b3d-dfbb-40ec-b79b-189ba6e3cba6 service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:34.713132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9391b3d-dfbb-40ec-b79b-189ba6e3cba6 service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:34.713413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9391b3d-dfbb-40ec-b79b-189ba6e3cba6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:34.722502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9391b3d-dfbb-40ec-b79b-189ba6e3cba6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:34.724121 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9391b3d-dfbb-40ec-b79b-189ba6e3cba6 service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:34.733789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 712/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:36.860796 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c92390-66af-4f95-aa60-1d811f4996e3 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:36.861297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c92390-66af-4f95-aa60-1d811f4996e3 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:36.861297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c92390-66af-4f95-aa60-1d811f4996e3 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:36.862622 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:36.867850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57714a-73ea-4abc-8e30-dea4f61bfee9 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:36.868305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57714a-73ea-4abc-8e30-dea4f61bfee9 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:36.868439 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57714a-73ea-4abc-8e30-dea4f61bfee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:36.887608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-4a57714a-73ea-4abc-8e30-dea4f61bfee9 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:36.888791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-4a57714a-73ea-4abc-8e30-dea4f61bfee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:36.896380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:36.934051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c92390-66af-4f95-aa60-1d811f4996e3 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:36.944773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c92390-66af-4f95-aa60-1d811f4996e3 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:36.948343 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c8da0-e368-4b51-846a-30f61e80e722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:36.949468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c8da0-e368-4b51-846a-30f61e80e722 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:36.949747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c8da0-e368-4b51-846a-30f61e80e722 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:36.950019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c8da0-e368-4b51-846a-30f61e80e722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:36.958745 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c8da0-e368-4b51-846a-30f61e80e722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:36.960219 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c8da0-e368-4b51-846a-30f61e80e722 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:36.972213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 730/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:39.324234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f9e79-ec03-42ca-b762-74205dc9b92f None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:39.324778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f9e79-ec03-42ca-b762-74205dc9b92f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:39.324778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f9e79-ec03-42ca-b762-74205dc9b92f None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:39.326127 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:39.331349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8860b-abeb-4344-a300-0a39659e9aee None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:39.331791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8860b-abeb-4344-a300-0a39659e9aee None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:39.331968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8860b-abeb-4344-a300-0a39659e9aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:39.350169 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-dde8860b-abeb-4344-a300-0a39659e9aee None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:39.351392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-dde8860b-abeb-4344-a300-0a39659e9aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:39.359615 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:39.396707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f9e79-ec03-42ca-b762-74205dc9b92f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:39.407899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f9e79-ec03-42ca-b762-74205dc9b92f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:39.411321 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb0019-2460-49c4-99f3-6d2985732c1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:39.411950 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb0019-2460-49c4-99f3-6d2985732c1c service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:39.412230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb0019-2460-49c4-99f3-6d2985732c1c service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:39.412507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb0019-2460-49c4-99f3-6d2985732c1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:39.421756 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb0019-2460-49c4-99f3-6d2985732c1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:39.423203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb0019-2460-49c4-99f3-6d2985732c1c service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:39.432283 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 715/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:41.986537 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c425a-9e90-4bd5-bac5-c5fdd839a185 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:41.987103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c425a-9e90-4bd5-bac5-c5fdd839a185 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:41.987103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c425a-9e90-4bd5-bac5-c5fdd839a185 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:41.988524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:41.993881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6301a-c653-47fd-91a6-39d3c6d972a3 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:41.994281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6301a-c653-47fd-91a6-39d3c6d972a3 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:41.994449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6301a-c653-47fd-91a6-39d3c6d972a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:42.013287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-fbd6301a-c653-47fd-91a6-39d3c6d972a3 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:42.014482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-fbd6301a-c653-47fd-91a6-39d3c6d972a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:42.022311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:42.059965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c425a-9e90-4bd5-bac5-c5fdd839a185 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:42.070561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c425a-9e90-4bd5-bac5-c5fdd839a185 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:42.074665 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9b988-9ad5-47cd-a0cf-1135aadc8aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:42.075328 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9b988-9ad5-47cd-a0cf-1135aadc8aea service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:42.075610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9b988-9ad5-47cd-a0cf-1135aadc8aea service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:42.075882 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9b988-9ad5-47cd-a0cf-1135aadc8aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:42.084719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9b988-9ad5-47cd-a0cf-1135aadc8aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:42.086173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9b988-9ad5-47cd-a0cf-1135aadc8aea service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:42.096205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 733/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:44.529737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283d146-c148-44f2-8af9-7c4c5ff4a7ab None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:44.530167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283d146-c148-44f2-8af9-7c4c5ff4a7ab None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:44.530318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283d146-c148-44f2-8af9-7c4c5ff4a7ab None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:44.531718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:44.537089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95e931-d6e4-4f94-abe0-9438addda292 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:44.537522 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95e931-d6e4-4f94-abe0-9438addda292 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:44.537627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95e931-d6e4-4f94-abe0-9438addda292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:44.555170 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-cc95e931-d6e4-4f94-abe0-9438addda292 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:44.556355 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-cc95e931-d6e4-4f94-abe0-9438addda292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:44.564564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:44.604275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283d146-c148-44f2-8af9-7c4c5ff4a7ab None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:44.615011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283d146-c148-44f2-8af9-7c4c5ff4a7ab None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:44.618512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1fd32-a675-4f5d-bad6-6beeaac963bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:44.619177 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1fd32-a675-4f5d-bad6-6beeaac963bf service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:44.619460 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1fd32-a675-4f5d-bad6-6beeaac963bf service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:44.619737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1fd32-a675-4f5d-bad6-6beeaac963bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:44.629219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1fd32-a675-4f5d-bad6-6beeaac963bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:44.630618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1fd32-a675-4f5d-bad6-6beeaac963bf service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:44.640356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 718/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:46.960829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fa77d-0e42-4c83-aadb-75f7fa8a5db7 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:46.961406 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fa77d-0e42-4c83-aadb-75f7fa8a5db7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:46.961406 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fa77d-0e42-4c83-aadb-75f7fa8a5db7 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:46.962762 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:46.968382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb907c47-7a55-41bb-948e-59891a41f06f None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:46.968851 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb907c47-7a55-41bb-948e-59891a41f06f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:46.968953 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb907c47-7a55-41bb-948e-59891a41f06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:46.986773 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-bb907c47-7a55-41bb-948e-59891a41f06f None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:46.987994 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-bb907c47-7a55-41bb-948e-59891a41f06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:46.996279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:47.034292 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fa77d-0e42-4c83-aadb-75f7fa8a5db7 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:47.044999 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fa77d-0e42-4c83-aadb-75f7fa8a5db7 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:47.048454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083a5d1-326c-471b-846b-422447b8bd02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:47.049082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083a5d1-326c-471b-846b-422447b8bd02 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:47.049369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083a5d1-326c-471b-846b-422447b8bd02 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:47.049644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083a5d1-326c-471b-846b-422447b8bd02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:47.059320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083a5d1-326c-471b-846b-422447b8bd02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:47.060778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083a5d1-326c-471b-846b-422447b8bd02 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:47.070240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 736/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:49.354303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc30ca-8bc8-4236-aee7-0fb526e24b8b None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:49.354976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc30ca-8bc8-4236-aee7-0fb526e24b8b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:49.354976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc30ca-8bc8-4236-aee7-0fb526e24b8b None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:49.356399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:49.361788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0099752-0473-4fdf-86d1-6ad98e7eb81b None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:49.362191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0099752-0473-4fdf-86d1-6ad98e7eb81b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:49.362346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0099752-0473-4fdf-86d1-6ad98e7eb81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:49.368380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=70 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:47:49.381666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-b0099752-0473-4fdf-86d1-6ad98e7eb81b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:49.382845 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-b0099752-0473-4fdf-86d1-6ad98e7eb81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:49.391732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:49.429269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc30ca-8bc8-4236-aee7-0fb526e24b8b None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:49.435338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=64 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:47:49.441712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc30ca-8bc8-4236-aee7-0fb526e24b8b None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:49.445263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a984ed-117e-48db-9fc1-1bd8d486508d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:49.445875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a984ed-117e-48db-9fc1-1bd8d486508d service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:49.446155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a984ed-117e-48db-9fc1-1bd8d486508d service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:49.446428 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a984ed-117e-48db-9fc1-1bd8d486508d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:49.455380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a984ed-117e-48db-9fc1-1bd8d486508d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:49.456784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a984ed-117e-48db-9fc1-1bd8d486508d service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:49.466186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 721/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:51.726765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775601d-7083-44a1-9742-d2fb4e95ea17 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:51.727352 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775601d-7083-44a1-9742-d2fb4e95ea17 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:51.727352 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775601d-7083-44a1-9742-d2fb4e95ea17 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:51.728634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:47:51.734067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c2ea1-18d4-4209-b023-9b03275ed10a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:51.734504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c2ea1-18d4-4209-b023-9b03275ed10a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:51.734617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c2ea1-18d4-4209-b023-9b03275ed10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:51.752514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-215c2ea1-18d4-4209-b023-9b03275ed10a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:51.753704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-215c2ea1-18d4-4209-b023-9b03275ed10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:51.762410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:51.799310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775601d-7083-44a1-9742-d2fb4e95ea17 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:51.810014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775601d-7083-44a1-9742-d2fb4e95ea17 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:51.813520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ede09-d0f6-4cfa-abd6-82e8ca25c23f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:51.814141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ede09-d0f6-4cfa-abd6-82e8ca25c23f service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:51.814422 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ede09-d0f6-4cfa-abd6-82e8ca25c23f service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:51.814692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ede09-d0f6-4cfa-abd6-82e8ca25c23f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:51.824043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ede09-d0f6-4cfa-abd6-82e8ca25c23f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:51.825641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ede09-d0f6-4cfa-abd6-82e8ca25c23f service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:51.835097 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 739/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:54.121836 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401897f7-9719-422f-9c5c-35fcc67f0d3e None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:54.122375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401897f7-9719-422f-9c5c-35fcc67f0d3e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:54.122375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401897f7-9719-422f-9c5c-35fcc67f0d3e None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:54.123769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:47:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:54.129274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9401844-62e1-4c69-91ef-61a1b76b9fd9 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:54.129724 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9401844-62e1-4c69-91ef-61a1b76b9fd9 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:54.129824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9401844-62e1-4c69-91ef-61a1b76b9fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:54.147826 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-d9401844-62e1-4c69-91ef-61a1b76b9fd9 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:54.149029 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d9401844-62e1-4c69-91ef-61a1b76b9fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:54.156907 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:54.193732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401897f7-9719-422f-9c5c-35fcc67f0d3e None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:54.204265 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401897f7-9719-422f-9c5c-35fcc67f0d3e None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:54.208048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63b40e-11ce-4e85-ac83-0e5bbdbe9fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:54.208686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63b40e-11ce-4e85-ac83-0e5bbdbe9fca service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:54.208970 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63b40e-11ce-4e85-ac83-0e5bbdbe9fca service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:54.209246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63b40e-11ce-4e85-ac83-0e5bbdbe9fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:54.218367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63b40e-11ce-4e85-ac83-0e5bbdbe9fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:54.219812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63b40e-11ce-4e85-ac83-0e5bbdbe9fca service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:54.228935 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 724/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:56.464181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475cee2-84de-46ba-8efc-3aad8796916a None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:56.464726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475cee2-84de-46ba-8efc-3aad8796916a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:56.464835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475cee2-84de-46ba-8efc-3aad8796916a None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:56.466384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:47:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:56.471828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52527a-feb5-4745-b9fc-900f389f30fa None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:56.472258 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52527a-feb5-4745-b9fc-900f389f30fa None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:56.472392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52527a-feb5-4745-b9fc-900f389f30fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:56.492692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-2b52527a-feb5-4745-b9fc-900f389f30fa None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:56.494784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2b52527a-feb5-4745-b9fc-900f389f30fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:56.505228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:56.541806 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475cee2-84de-46ba-8efc-3aad8796916a None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:56.552969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475cee2-84de-46ba-8efc-3aad8796916a None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:56.556448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e36e48-bac2-432a-923b-da4cc83d455e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:56.557073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e36e48-bac2-432a-923b-da4cc83d455e service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:56.557356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e36e48-bac2-432a-923b-da4cc83d455e service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:56.557777 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e36e48-bac2-432a-923b-da4cc83d455e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:56.566619 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e36e48-bac2-432a-923b-da4cc83d455e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:56.568091 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e36e48-bac2-432a-923b-da4cc83d455e service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:56.577623 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 742/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:47:58.814200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d806-bba8-469b-8c7b-eea6f4ed43f1 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:58.814711 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d806-bba8-469b-8c7b-eea6f4ed43f1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:58.814711 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d806-bba8-469b-8c7b-eea6f4ed43f1 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:58.816082 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:47:58.821473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360a9e9-5333-4174-95e1-3fc034c58c93 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:58.821923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360a9e9-5333-4174-95e1-3fc034c58c93 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:58.822062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360a9e9-5333-4174-95e1-3fc034c58c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:58.840123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-7360a9e9-5333-4174-95e1-3fc034c58c93 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:47:58.841313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-7360a9e9-5333-4174-95e1-3fc034c58c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:47:58.849109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:47:58.885397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30d806-bba8-469b-8c7b-eea6f4ed43f1 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:47:58.896698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30d806-bba8-469b-8c7b-eea6f4ed43f1 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:47:58.900227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1281838-6990-4b18-a605-6a34931c2115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:47:58.900859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1281838-6990-4b18-a605-6a34931c2115 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:47:58.901142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1281838-6990-4b18-a605-6a34931c2115 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:47:58.901916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1281838-6990-4b18-a605-6a34931c2115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:47:58.910773 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1281838-6990-4b18-a605-6a34931c2115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:47:58.912212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1281838-6990-4b18-a605-6a34931c2115 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:47:58.923098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 727/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:47:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:48:08.835187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=20 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:48:08.917790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=23 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:48:32.146939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6397ef4-20c6-4084-aac2-b470d50c32ba None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:32.147550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6397ef4-20c6-4084-aac2-b470d50c32ba None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:32.147550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6397ef4-20c6-4084-aac2-b470d50c32ba None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:32.148911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 744/1471] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:48:32.154846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec0c5-e209-45fc-bb01-e2521a0f6b77 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:32.155344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec0c5-e209-45fc-bb01-e2521a0f6b77 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:32.155459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec0c5-e209-45fc-bb01-e2521a0f6b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:32.173500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-722ec0c5-e209-45fc-bb01-e2521a0f6b77 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:48:32.174692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-722ec0c5-e209-45fc-bb01-e2521a0f6b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:48:32.182594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 728/1472] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:48:32.229288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb90974-181d-40a0-9a6e-a7c70de53a1b None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:32.229579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb90974-181d-40a0-9a6e-a7c70de53a1b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:32.229862 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb90974-181d-40a0-9a6e-a7c70de53a1b None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:32.231273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 745/1473] 10.222.0.52 () {54 vars in 962 bytes} [Thu May 19 13:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:48:32.236011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d149945-269e-4c08-9bc0-9170131be42f None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:32.236309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d149945-269e-4c08-9bc0-9170131be42f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:32.236590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d149945-269e-4c08-9bc0-9170131be42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:32.254770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-9d149945-269e-4c08-9bc0-9170131be42f None None] MFA Rules not processed for user `666ade4ce1fe447487998d086779389b`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:48:32.255976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9d149945-269e-4c08-9bc0-9170131be42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:48:32.262835 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 729/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Thu May 19 13:48:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:48:32.269634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb90974-181d-40a0-9a6e-a7c70de53a1b None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:48:32.280835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb90974-181d-40a0-9a6e-a7c70de53a1b None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:48:32.284347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bdc61-91d9-4e21-ad77-1c533668b674 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '666ade4ce1fe447487998d086779389b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:48:32.284967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bdc61-91d9-4e21-ad77-1c533668b674 service glance] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:32.285245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bdc61-91d9-4e21-ad77-1c533668b674 service glance] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:32.285518 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bdc61-91d9-4e21-ad77-1c533668b674 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:32.294328 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bdc61-91d9-4e21-ad77-1c533668b674 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:48:32.295797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bdc61-91d9-4e21-ad77-1c533668b674 service glance] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:48:32.305151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 746/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:48:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:48:34.636982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe3b46-d642-4e8a-9dde-74a84918228c None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:34.637531 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe3b46-d642-4e8a-9dde-74a84918228c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:34.637531 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe3b46-d642-4e8a-9dde-74a84918228c None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:34.638943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 730/1476] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:48:34.644181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37ecc3-071e-4daa-8a05-26204cbf4a48 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:34.644601 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37ecc3-071e-4daa-8a05-26204cbf4a48 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:34.644741 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37ecc3-071e-4daa-8a05-26204cbf4a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:34.663125 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-1f37ecc3-071e-4daa-8a05-26204cbf4a48 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:48:34.664264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1f37ecc3-071e-4daa-8a05-26204cbf4a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:48:34.672152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 747/1477] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:48:34.695107 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe3b46-d642-4e8a-9dde-74a84918228c None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:48:34.708003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe3b46-d642-4e8a-9dde-74a84918228c None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:48:34.712710 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f020682-6183-4ce0-ae19-c57296907fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:48:34.714048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f020682-6183-4ce0-ae19-c57296907fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:34.714329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f020682-6183-4ce0-ae19-c57296907fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:34.714604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f020682-6183-4ce0-ae19-c57296907fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:34.715396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f020682-6183-4ce0-ae19-c57296907fb1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:48:34.717146 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: WARNING py.warnings [None req-4f020682-6183-4ce0-ae19-c57296907fb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:48:34.717146 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: warnings.warn(msg) May 19 13:48:34.717146 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: May 19 13:48:34.718268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f020682-6183-4ce0-ae19-c57296907fb1 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:48:34.745051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 731/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Thu May 19 13:48:34 2022] GET /identity/v3/projects => generated 2553 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:48:37.007116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbea5c7-fb81-49d7-80ee-e8bd38bfc0e3 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:37.007768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbea5c7-fb81-49d7-80ee-e8bd38bfc0e3 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:37.007768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbea5c7-fb81-49d7-80ee-e8bd38bfc0e3 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:37.009050 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 748/1479] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:48:37.014937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e3027-29c9-4742-940c-31395369737d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:37.015369 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e3027-29c9-4742-940c-31395369737d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:37.015471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e3027-29c9-4742-940c-31395369737d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:37.033994 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6d3e3027-29c9-4742-940c-31395369737d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:48:37.035298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6d3e3027-29c9-4742-940c-31395369737d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:48:37.043632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 732/1480] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:48:37.088205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbea5c7-fb81-49d7-80ee-e8bd38bfc0e3 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:48:37.102101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbea5c7-fb81-49d7-80ee-e8bd38bfc0e3 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:48:37.107879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0525a-b3d8-4381-9fc3-52039f3e83df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:48:37.108506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0525a-b3d8-4381-9fc3-52039f3e83df service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:37.108788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0525a-b3d8-4381-9fc3-52039f3e83df service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:37.109063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0525a-b3d8-4381-9fc3-52039f3e83df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:37.117069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0525a-b3d8-4381-9fc3-52039f3e83df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:48:37.118633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0525a-b3d8-4381-9fc3-52039f3e83df service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:48:37.128877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 749/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:48:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:48:39.545511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c8d61-4fe0-435f-bbc8-a399b908c323 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:39.546078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c8d61-4fe0-435f-bbc8-a399b908c323 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:39.546078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c8d61-4fe0-435f-bbc8-a399b908c323 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:39.547469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 733/1482] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:48:39.552773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c11e4-5e4b-48f8-adb8-28d0402268e7 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:39.553197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c11e4-5e4b-48f8-adb8-28d0402268e7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:39.553340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c11e4-5e4b-48f8-adb8-28d0402268e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:39.571631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-a63c11e4-5e4b-48f8-adb8-28d0402268e7 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:48:39.573742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a63c11e4-5e4b-48f8-adb8-28d0402268e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:48:39.581589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 750/1483] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:48:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:48:39.617582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c8d61-4fe0-435f-bbc8-a399b908c323 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:48:39.629068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c8d61-4fe0-435f-bbc8-a399b908c323 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:48:39.632604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1084e24-1208-46a2-8917-36c64a56e27e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:48:39.633242 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1084e24-1208-46a2-8917-36c64a56e27e service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:39.633522 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1084e24-1208-46a2-8917-36c64a56e27e service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:39.633794 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1084e24-1208-46a2-8917-36c64a56e27e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:39.643193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1084e24-1208-46a2-8917-36c64a56e27e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:48:39.644712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1084e24-1208-46a2-8917-36c64a56e27e service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:48:39.654167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 734/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:48:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:48:41.945392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15442a-3430-467f-84d1-7ce224a47a0b None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:41.945969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15442a-3430-467f-84d1-7ce224a47a0b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:41.945969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15442a-3430-467f-84d1-7ce224a47a0b None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:41.947992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 751/1485] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:48:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:48:41.953300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260c2bb-64e4-4053-aac7-85b82f032055 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:41.953748 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260c2bb-64e4-4053-aac7-85b82f032055 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:41.953876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260c2bb-64e4-4053-aac7-85b82f032055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:41.972112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-7260c2bb-64e4-4053-aac7-85b82f032055 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:48:41.973315 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-7260c2bb-64e4-4053-aac7-85b82f032055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:48:41.981438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 735/1486] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:48:42.018734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15442a-3430-467f-84d1-7ce224a47a0b None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:48:42.029857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15442a-3430-467f-84d1-7ce224a47a0b None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:48:42.033351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da876b0d-cf27-4750-8d78-5b5016e146cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:48:42.033971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da876b0d-cf27-4750-8d78-5b5016e146cb service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:42.034253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da876b0d-cf27-4750-8d78-5b5016e146cb service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:42.034526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da876b0d-cf27-4750-8d78-5b5016e146cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:42.043345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da876b0d-cf27-4750-8d78-5b5016e146cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:48:42.044761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da876b0d-cf27-4750-8d78-5b5016e146cb service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:48:42.053962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 752/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:48:44.353293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf2b02-8026-47bf-ae2e-2fea18192cf8 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:44.354014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf2b02-8026-47bf-ae2e-2fea18192cf8 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:44.354014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf2b02-8026-47bf-ae2e-2fea18192cf8 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:44.355397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 736/1488] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:48:44.360520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb7acd-4ab5-4fd0-9b5e-d8f3d852f26e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:44.360952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb7acd-4ab5-4fd0-9b5e-d8f3d852f26e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:44.361067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb7acd-4ab5-4fd0-9b5e-d8f3d852f26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:44.379203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-07cb7acd-4ab5-4fd0-9b5e-d8f3d852f26e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:48:44.380382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-07cb7acd-4ab5-4fd0-9b5e-d8f3d852f26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:48:44.388845 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 753/1489] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:48:44.428165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf2b02-8026-47bf-ae2e-2fea18192cf8 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:48:44.438709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf2b02-8026-47bf-ae2e-2fea18192cf8 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:48:44.442284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583bf40-5b68-4eb0-b5b9-cb796f8212fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:48:44.442939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583bf40-5b68-4eb0-b5b9-cb796f8212fb service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:44.443224 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583bf40-5b68-4eb0-b5b9-cb796f8212fb service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:44.443498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583bf40-5b68-4eb0-b5b9-cb796f8212fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:44.452836 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583bf40-5b68-4eb0-b5b9-cb796f8212fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:48:44.454251 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583bf40-5b68-4eb0-b5b9-cb796f8212fb service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:48:44.464220 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 737/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:48:46.893667 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99cbd3-c1f8-47c4-80d5-be022b2563a5 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:46.894236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99cbd3-c1f8-47c4-80d5-be022b2563a5 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:46.894236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99cbd3-c1f8-47c4-80d5-be022b2563a5 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:46.895655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 754/1491] 10.222.0.52 () {54 vars in 926 bytes} [Thu May 19 13:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:48:46.901559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801d475-27e0-4efa-9013-c0fd2fc2ce0b None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:46.901990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801d475-27e0-4efa-9013-c0fd2fc2ce0b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:46.902138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801d475-27e0-4efa-9013-c0fd2fc2ce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:46.919831 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-5801d475-27e0-4efa-9013-c0fd2fc2ce0b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:48:46.921011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5801d475-27e0-4efa-9013-c0fd2fc2ce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:48:46.929075 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 738/1492] 10.222.0.52 () {58 vars in 1023 bytes} [Thu May 19 13:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:48:46.967816 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99cbd3-c1f8-47c4-80d5-be022b2563a5 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:48:46.979579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99cbd3-c1f8-47c4-80d5-be022b2563a5 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:48:46.983726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a3026-be76-4b7f-b629-7f124d4a2df8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:48:46.984366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a3026-be76-4b7f-b629-7f124d4a2df8 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:48:46.984654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a3026-be76-4b7f-b629-7f124d4a2df8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:48:46.984933 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a3026-be76-4b7f-b629-7f124d4a2df8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:48:46.994901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a3026-be76-4b7f-b629-7f124d4a2df8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:48:46.996392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a3026-be76-4b7f-b629-7f124d4a2df8 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:48:47.005935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 755/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:48:56.914498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=33 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:48:57.001905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=33 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:50:27.135244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3eae6-d08f-4eba-b2fe-fe2a54fb3c9a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.135847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3eae6-d08f-4eba-b2fe-fe2a54fb3c9a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.135847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3eae6-d08f-4eba-b2fe-fe2a54fb3c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.153857 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-dae3eae6-d08f-4eba-b2fe-fe2a54fb3c9a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:50:27.155087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-dae3eae6-d08f-4eba-b2fe-fe2a54fb3c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:50:27.164435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 739/1494] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:50:27.173691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80831b66-aa82-47de-848f-a72b7242f8a5 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.174181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80831b66-aa82-47de-848f-a72b7242f8a5 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.174282 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80831b66-aa82-47de-848f-a72b7242f8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.193162 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-80831b66-aa82-47de-848f-a72b7242f8a5 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:50:27.194390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-80831b66-aa82-47de-848f-a72b7242f8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:50:27.202498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 756/1495] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:50:27.207003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3eae6-d08f-4eba-b2fe-fe2a54fb3c9a None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:50:27.216579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3eae6-d08f-4eba-b2fe-fe2a54fb3c9a None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:50:27.221051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149f7d7-fd0c-470b-b3ab-b547b6c3da95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:50:27.221696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149f7d7-fd0c-470b-b3ab-b547b6c3da95 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.221971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149f7d7-fd0c-470b-b3ab-b547b6c3da95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.222249 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149f7d7-fd0c-470b-b3ab-b547b6c3da95 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.223033 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149f7d7-fd0c-470b-b3ab-b547b6c3da95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:50:27.223338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149f7d7-fd0c-470b-b3ab-b547b6c3da95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:50:27.224333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: WARNING py.warnings [None req-7149f7d7-fd0c-470b-b3ab-b547b6c3da95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 13:50:27.224333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: warnings.warn(msg) May 19 13:50:27.224333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: May 19 13:50:27.225198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149f7d7-fd0c-470b-b3ab-b547b6c3da95 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:50:27.237211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 740/1496] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:50:27.245801 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80831b66-aa82-47de-848f-a72b7242f8a5 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:50:27.255188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80831b66-aa82-47de-848f-a72b7242f8a5 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:50:27.259844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5234ea-1d39-4809-9723-a8fa1389e0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:50:27.260488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5234ea-1d39-4809-9723-a8fa1389e0d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.260776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5234ea-1d39-4809-9723-a8fa1389e0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.261053 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5234ea-1d39-4809-9723-a8fa1389e0d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.261753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5234ea-1d39-4809-9723-a8fa1389e0d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:50:27.263585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: WARNING py.warnings [None req-0b5234ea-1d39-4809-9723-a8fa1389e0d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:50:27.263585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: warnings.warn(msg) May 19 13:50:27.263585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: May 19 13:50:27.264780 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5234ea-1d39-4809-9723-a8fa1389e0d4 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:50:27.300392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 757/1497] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:50:27 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:50:27.306459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149f7d7-fd0c-470b-b3ab-b547b6c3da95 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:50:27.316127 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149f7d7-fd0c-470b-b3ab-b547b6c3da95 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:50:27.372058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9f76c-398d-4cc4-8e5e-85bfcb1acbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:50:27.372738 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9f76c-398d-4cc4-8e5e-85bfcb1acbe7 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.373026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9f76c-398d-4cc4-8e5e-85bfcb1acbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.373306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9f76c-398d-4cc4-8e5e-85bfcb1acbe7 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.374001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9f76c-398d-4cc4-8e5e-85bfcb1acbe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:50:27.375036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: WARNING py.warnings [None req-2dc9f76c-398d-4cc4-8e5e-85bfcb1acbe7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:50:27.375036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: warnings.warn(msg) May 19 13:50:27.375036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: May 19 13:50:27.376494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9f76c-398d-4cc4-8e5e-85bfcb1acbe7 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:50:27.398590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 741/1498] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:50:27 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:50:27.402639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5234ea-1d39-4809-9723-a8fa1389e0d4 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:50:27.412958 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5234ea-1d39-4809-9723-a8fa1389e0d4 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:50:27.418000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3eb486-0dfa-4eae-afab-a822a4218597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:50:27.418643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eb486-0dfa-4eae-afab-a822a4218597 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.418952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eb486-0dfa-4eae-afab-a822a4218597 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.419278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eb486-0dfa-4eae-afab-a822a4218597 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.419998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3eb486-0dfa-4eae-afab-a822a4218597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:50:27.421093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: WARNING py.warnings [None req-da3eb486-0dfa-4eae-afab-a822a4218597 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 13:50:27.421093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: warnings.warn(msg) May 19 13:50:27.421093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: May 19 13:50:27.422169 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3eb486-0dfa-4eae-afab-a822a4218597 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:50:27.435613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 758/1499] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:50:27.439804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9f76c-398d-4cc4-8e5e-85bfcb1acbe7 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:50:27.450177 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9f76c-398d-4cc4-8e5e-85bfcb1acbe7 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:50:27.454803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07e5d9-427f-4ae2-95e0-e8971897081b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:50:27.455495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07e5d9-427f-4ae2-95e0-e8971897081b admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.455785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07e5d9-427f-4ae2-95e0-e8971897081b admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.456067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07e5d9-427f-4ae2-95e0-e8971897081b admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.456793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07e5d9-427f-4ae2-95e0-e8971897081b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:50:27.457797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: WARNING py.warnings [None req-1b07e5d9-427f-4ae2-95e0-e8971897081b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 13:50:27.457797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: warnings.warn(msg) May 19 13:50:27.457797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: May 19 13:50:27.458675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07e5d9-427f-4ae2-95e0-e8971897081b admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:50:27.472623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 742/1500] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:50:27.477349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3eb486-0dfa-4eae-afab-a822a4218597 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:50:27.487078 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3eb486-0dfa-4eae-afab-a822a4218597 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:50:27.491521 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e93a3-44d7-4ebd-a741-259cb7fdddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:50:27.492409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e93a3-44d7-4ebd-a741-259cb7fdddf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.492685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e93a3-44d7-4ebd-a741-259cb7fdddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.492957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e93a3-44d7-4ebd-a741-259cb7fdddf5 admin admin] PATH_INFO: `/v3/projects/f542eabb21544d13a3bd1a92746e528a/users/ea619240aa0a4b87bb9ece69edcb9925/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.507435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e93a3-44d7-4ebd-a741-259cb7fdddf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f542eabb21544d13a3bd1a92746e528a, user_id=ea619240aa0a4b87bb9ece69edcb9925, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:50:27.508487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: WARNING py.warnings [None req-a77e93a3-44d7-4ebd-a741-259cb7fdddf5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:50:27.508487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: warnings.warn(msg) May 19 13:50:27.508487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: May 19 13:50:27.510386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e93a3-44d7-4ebd-a741-259cb7fdddf5 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:50:27.523774 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 759/1501] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:50:27 2022] PUT /identity/v3/projects/f542eabb21544d13a3bd1a92746e528a/users/ea619240aa0a4b87bb9ece69edcb9925/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:50:27.535307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4772cb7-2cbf-4deb-b637-3e6a7a6ca4c4 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.535602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4772cb7-2cbf-4deb-b637-3e6a7a6ca4c4 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.535885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4772cb7-2cbf-4deb-b637-3e6a7a6ca4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.561990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-c4772cb7-2cbf-4deb-b637-3e6a7a6ca4c4 None None] MFA Rules not processed for user `ea619240aa0a4b87bb9ece69edcb9925`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:50:27.563238 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c4772cb7-2cbf-4deb-b637-3e6a7a6ca4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:50:27.625899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 743/1502] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:50:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:50:27.639343 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1f61c-5e52-4a9b-8bca-1816a213f378 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.639630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1f61c-5e52-4a9b-8bca-1816a213f378 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.639905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1f61c-5e52-4a9b-8bca-1816a213f378 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.641264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 760/1503] 10.222.0.52 () {54 vars in 963 bytes} [Thu May 19 13:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:50:27.646106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4f5b0-5899-49dd-8726-aa3f1280f43d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.646403 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4f5b0-5899-49dd-8726-aa3f1280f43d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.646688 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4f5b0-5899-49dd-8726-aa3f1280f43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.665812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-e2f4f5b0-5899-49dd-8726-aa3f1280f43d None None] MFA Rules not processed for user `1ed0973226a2446ebf7c0cc3ee04e446`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:50:27.666973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e2f4f5b0-5899-49dd-8726-aa3f1280f43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:50:27.708766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:50:27.717137 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 744/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Thu May 19 13:50:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:50:27.724915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1f61c-5e52-4a9b-8bca-1816a213f378 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:50:27.735513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1f61c-5e52-4a9b-8bca-1816a213f378 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:50:27.739266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ede86-57ba-4304-b772-bf8330c3466a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:50:27.739895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ede86-57ba-4304-b772-bf8330c3466a service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.740181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ede86-57ba-4304-b772-bf8330c3466a service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.740462 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ede86-57ba-4304-b772-bf8330c3466a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.748807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ede86-57ba-4304-b772-bf8330c3466a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:50:27.750228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ede86-57ba-4304-b772-bf8330c3466a service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:50:27.759979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 761/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:50:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:50:27.800871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954ec33-4576-403c-9e99-4055ca04df5c None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.801169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954ec33-4576-403c-9e99-4055ca04df5c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.801453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954ec33-4576-403c-9e99-4055ca04df5c None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.802729 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 745/1506] 10.222.0.52 () {52 vars in 829 bytes} [Thu May 19 13:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:50:27.806481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ede86-57ba-4304-b772-bf8330c3466a service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:50:27.866847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ede86-57ba-4304-b772-bf8330c3466a service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:50:27.871389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:50:27.872252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.872536 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.872814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] PATH_INFO: `/v3/users/ea619240aa0a4b87bb9ece69edcb9925` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:27.873987 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea619240aa0a4b87bb9ece69edcb9925)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:50:27.875032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: WARNING py.warnings [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:50:27.875032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: warnings.warn(msg) May 19 13:50:27.875032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: May 19 13:50:27.876483 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:50:27.941532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea619240aa0a4b87bb9ece69edcb9925'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:50:27.951867 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea619240aa0a4b87bb9ece69edcb9925'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:50:27.964554 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea619240aa0a4b87bb9ece69edcb9925'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:50:27.965286 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea619240aa0a4b87bb9ece69edcb9925'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:50:27.965895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea619240aa0a4b87bb9ece69edcb9925'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:50:27.983263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:50:27.992092 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 762/1507] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:50:27 2022] DELETE /identity/v3/users/ea619240aa0a4b87bb9ece69edcb9925 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:50:27.996607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187a93b-f281-44c9-9b05-1949c9af4312 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:27.996903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187a93b-f281-44c9-9b05-1949c9af4312 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:27.997182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187a93b-f281-44c9-9b05-1949c9af4312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:28.014940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-1187a93b-f281-44c9-9b05-1949c9af4312 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:50:28.016129 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1187a93b-f281-44c9-9b05-1949c9af4312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:50:28.074690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 746/1508] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:50:28.084823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:50:28.086254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:50:28.143290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31881e47-36dc-43cb-9276-be3488a07915 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:50:28.146819 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f199d-d5a4-47c9-913d-328853a4b924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:50:28.147488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f199d-d5a4-47c9-913d-328853a4b924 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:28.147772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f199d-d5a4-47c9-913d-328853a4b924 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:28.148161 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f199d-d5a4-47c9-913d-328853a4b924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:28.157205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f199d-d5a4-47c9-913d-328853a4b924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:50:28.158664 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f199d-d5a4-47c9-913d-328853a4b924 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:50:28.168473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 763/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:50:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:50:28.529427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187a93b-f281-44c9-9b05-1949c9af4312 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:50:28.530741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1187a93b-f281-44c9-9b05-1949c9af4312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:50:28.543356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187a93b-f281-44c9-9b05-1949c9af4312 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:50:28.548623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869097e5-c0ce-429f-af8d-3c1050e3ffa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:50:28.549481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869097e5-c0ce-429f-af8d-3c1050e3ffa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:28.549759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869097e5-c0ce-429f-af8d-3c1050e3ffa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:28.550031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869097e5-c0ce-429f-af8d-3c1050e3ffa2 admin admin] PATH_INFO: `/v3/projects/f542eabb21544d13a3bd1a92746e528a` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:28.551254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869097e5-c0ce-429f-af8d-3c1050e3ffa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f542eabb21544d13a3bd1a92746e528a)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:50:28.552269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: WARNING py.warnings [None req-869097e5-c0ce-429f-af8d-3c1050e3ffa2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:50:28.552269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: warnings.warn(msg) May 19 13:50:28.552269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: May 19 13:50:28.553463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869097e5-c0ce-429f-af8d-3c1050e3ffa2 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:50:28.614114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-869097e5-c0ce-429f-af8d-3c1050e3ffa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f542eabb21544d13a3bd1a92746e528a', 'request_id': 'req-869097e5-c0ce-429f-af8d-3c1050e3ffa2'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:50:28.628200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-869097e5-c0ce-429f-af8d-3c1050e3ffa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f542eabb21544d13a3bd1a92746e528a', 'request_id': 'req-869097e5-c0ce-429f-af8d-3c1050e3ffa2'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:50:28.637222 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-869097e5-c0ce-429f-af8d-3c1050e3ffa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f542eabb21544d13a3bd1a92746e528a', 'request_id': 'req-869097e5-c0ce-429f-af8d-3c1050e3ffa2'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:50:28.648098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-869097e5-c0ce-429f-af8d-3c1050e3ffa2 admin admin] The token cache is being invalidate because project f542eabb21544d13a3bd1a92746e528a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:50:28.648397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-869097e5-c0ce-429f-af8d-3c1050e3ffa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:50:28.649120 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-869097e5-c0ce-429f-af8d-3c1050e3ffa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:50:28.650644 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 747/1510] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:50:28 2022] DELETE /identity/v3/projects/f542eabb21544d13a3bd1a92746e528a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:50:38.164477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:50:38.645058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:52:12.683666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c552535-d9b3-4376-bc35-d812ec60e897 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:12.684188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c552535-d9b3-4376-bc35-d812ec60e897 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:12.684293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c552535-d9b3-4376-bc35-d812ec60e897 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:12.685606 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:52:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:52:12.690625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d925eb-7353-408f-a2bc-c7a85731a54d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:12.691081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d925eb-7353-408f-a2bc-c7a85731a54d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:12.691227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d925eb-7353-408f-a2bc-c7a85731a54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:12.710084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-67d925eb-7353-408f-a2bc-c7a85731a54d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:52:12.711343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-67d925eb-7353-408f-a2bc-c7a85731a54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:12.769640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:52:12.944655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c552535-d9b3-4376-bc35-d812ec60e897 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:52:12.946382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8c552535-d9b3-4376-bc35-d812ec60e897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:13.018827 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c552535-d9b3-4376-bc35-d812ec60e897 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:52:13.024162 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fbca1-04f1-4058-8218-a19fcd67db36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:52:13.024795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fbca1-04f1-4058-8218-a19fcd67db36 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:13.025102 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fbca1-04f1-4058-8218-a19fcd67db36 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:13.025395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fbca1-04f1-4058-8218-a19fcd67db36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:13.033378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5fbca1-04f1-4058-8218-a19fcd67db36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:52:13.034989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5fbca1-04f1-4058-8218-a19fcd67db36 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:52:13.045138 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 765/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:52:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:52:13.866715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa5aa1-0106-4ac7-acb3-8b9f547b9704 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:13.867305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa5aa1-0106-4ac7-acb3-8b9f547b9704 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:13.867305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa5aa1-0106-4ac7-acb3-8b9f547b9704 None None] PATH_INFO: `/v3` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:13.868588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 749/1514] 10.222.0.65 () {54 vars in 925 bytes} [Thu May 19 13:52:13 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:52:13.897988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cbb94-d79b-4159-866c-486d4de1e580 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:13.898503 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cbb94-d79b-4159-866c-486d4de1e580 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:13.898692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cbb94-d79b-4159-866c-486d4de1e580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:13.917452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-927cbb94-d79b-4159-866c-486d4de1e580 None None] MFA Rules not processed for user `b105f8163ce84a9ba725e6b04d348446`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:52:13.918668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-927cbb94-d79b-4159-866c-486d4de1e580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:13.943797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ab2b0-f53a-4183-afcf-c3b88a488f20 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:13.944088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ab2b0-f53a-4183-afcf-c3b88a488f20 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:13.944367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ab2b0-f53a-4183-afcf-c3b88a488f20 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:13.945734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 750/1515] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:52:13.951099 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ec5b8-b600-4cd2-9fe0-7cd25d2024fe None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:13.951392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ec5b8-b600-4cd2-9fe0-7cd25d2024fe None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:13.951676 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ec5b8-b600-4cd2-9fe0-7cd25d2024fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:13.970596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-486ec5b8-b600-4cd2-9fe0-7cd25d2024fe None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:52:13.971820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-486ec5b8-b600-4cd2-9fe0-7cd25d2024fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:13.980041 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 751/1516] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:52:13.985259 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 766/1517] 10.222.0.65 () {58 vars in 1015 bytes} [Thu May 19 13:52:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:52:14.131789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ec5b8-b600-4cd2-9fe0-7cd25d2024fe None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:52:14.144998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ec5b8-b600-4cd2-9fe0-7cd25d2024fe None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:52:14.150663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e4c55-2114-4b7c-bb64-9f6257cfbcc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:52:14.151471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e4c55-2114-4b7c-bb64-9f6257cfbcc6 service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:14.151759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e4c55-2114-4b7c-bb64-9f6257cfbcc6 service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:14.152037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e4c55-2114-4b7c-bb64-9f6257cfbcc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:14.160055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e4c55-2114-4b7c-bb64-9f6257cfbcc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:52:14.161641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e4c55-2114-4b7c-bb64-9f6257cfbcc6 service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:52:14.171020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 752/1518] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:52:23.111768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2372e96-8b05-42d5-9d5a-4d71ddecf910 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:23.112296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2372e96-8b05-42d5-9d5a-4d71ddecf910 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:23.112296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2372e96-8b05-42d5-9d5a-4d71ddecf910 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:23.113629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 767/1519] 10.222.0.52 () {54 vars in 960 bytes} [Thu May 19 13:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:52:23.119407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca51cba-e0c1-46fb-8608-f5f9caf6e0b4 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:23.119832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca51cba-e0c1-46fb-8608-f5f9caf6e0b4 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:23.119945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca51cba-e0c1-46fb-8608-f5f9caf6e0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:23.138361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-9ca51cba-e0c1-46fb-8608-f5f9caf6e0b4 None None] MFA Rules not processed for user `b105f8163ce84a9ba725e6b04d348446`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:52:23.139582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9ca51cba-e0c1-46fb-8608-f5f9caf6e0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:23.147967 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 753/1520] 10.222.0.52 () {58 vars in 1057 bytes} [Thu May 19 13:52:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:52:23.155258 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2372e96-8b05-42d5-9d5a-4d71ddecf910 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:52:23.168512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2372e96-8b05-42d5-9d5a-4d71ddecf910 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:52:23.174021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6cdc9-6842-4a33-b4b2-0f72509729fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b105f8163ce84a9ba725e6b04d348446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:52:23.174654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6cdc9-6842-4a33-b4b2-0f72509729fe service placement] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:23.174974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6cdc9-6842-4a33-b4b2-0f72509729fe service placement] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:23.175259 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6cdc9-6842-4a33-b4b2-0f72509729fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:23.184492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6cdc9-6842-4a33-b4b2-0f72509729fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:52:23.186076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6cdc9-6842-4a33-b4b2-0f72509729fe service placement] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:52:23.195616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 768/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:52:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:52:25.476741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbcdba-9dbb-46e6-83cc-146bef594339 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:25.477297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbcdba-9dbb-46e6-83cc-146bef594339 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:25.477297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbcdba-9dbb-46e6-83cc-146bef594339 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:25.479390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:52:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:52:25.484242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38822b48-228d-4f4a-904c-347b6664bb09 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:25.484676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38822b48-228d-4f4a-904c-347b6664bb09 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:25.484813 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38822b48-228d-4f4a-904c-347b6664bb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:25.503721 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-38822b48-228d-4f4a-904c-347b6664bb09 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:52:25.504959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-38822b48-228d-4f4a-904c-347b6664bb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:25.513731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:52:25.666891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbcdba-9dbb-46e6-83cc-146bef594339 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:52:25.681524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbcdba-9dbb-46e6-83cc-146bef594339 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:52:25.686881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a411b50-50b8-404d-8bd8-19b2131244f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:52:25.687521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a411b50-50b8-404d-8bd8-19b2131244f5 service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:25.687803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a411b50-50b8-404d-8bd8-19b2131244f5 service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:25.688076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a411b50-50b8-404d-8bd8-19b2131244f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:25.695951 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a411b50-50b8-404d-8bd8-19b2131244f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:52:25.698228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a411b50-50b8-404d-8bd8-19b2131244f5 service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:52:25.707405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 755/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:52:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:52:26.601418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c27924-d90f-4f97-b653-d6301fc644f6 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:26.601985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c27924-d90f-4f97-b653-d6301fc644f6 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:26.601985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c27924-d90f-4f97-b653-d6301fc644f6 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:26.603575 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:52:26.608493 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f7f96-6631-4ec0-9218-31ae6453bdc1 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:26.608896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f7f96-6631-4ec0-9218-31ae6453bdc1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:26.609063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f7f96-6631-4ec0-9218-31ae6453bdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:26.629248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-8c4f7f96-6631-4ec0-9218-31ae6453bdc1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:52:26.630459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-8c4f7f96-6631-4ec0-9218-31ae6453bdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:26.639213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:52:26.790043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c27924-d90f-4f97-b653-d6301fc644f6 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:52:26.804265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c27924-d90f-4f97-b653-d6301fc644f6 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:52:26.810182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc3a76-14c0-4ddc-b7a6-6c7427c6bdbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:52:26.810808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3a76-14c0-4ddc-b7a6-6c7427c6bdbf service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:26.811284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3a76-14c0-4ddc-b7a6-6c7427c6bdbf service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:26.811565 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3a76-14c0-4ddc-b7a6-6c7427c6bdbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:26.819598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc3a76-14c0-4ddc-b7a6-6c7427c6bdbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:52:26.821188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc3a76-14c0-4ddc-b7a6-6c7427c6bdbf service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:52:26.831426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 771/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:52:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:52:31.179948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ea258-5c04-4e81-8c73-0cc488556f1c None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:31.180486 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ea258-5c04-4e81-8c73-0cc488556f1c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:31.180486 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ea258-5c04-4e81-8c73-0cc488556f1c None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:31.182462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:52:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:52:31.187639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52d725-bac6-41dd-96a7-095f12d034c7 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:31.188070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52d725-bac6-41dd-96a7-095f12d034c7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:31.188206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52d725-bac6-41dd-96a7-095f12d034c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:31.206515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ae52d725-bac6-41dd-96a7-095f12d034c7 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:52:31.207729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ae52d725-bac6-41dd-96a7-095f12d034c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:31.215589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:52:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:52:31.252277 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ea258-5c04-4e81-8c73-0cc488556f1c None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:52:31.253991 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-313ea258-5c04-4e81-8c73-0cc488556f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:31.310563 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ea258-5c04-4e81-8c73-0cc488556f1c None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:52:31.314104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8f146-64be-4f23-b34b-9c97d6080064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:52:31.314749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8f146-64be-4f23-b34b-9c97d6080064 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:31.315076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8f146-64be-4f23-b34b-9c97d6080064 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:31.315357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8f146-64be-4f23-b34b-9c97d6080064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:31.324965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8f146-64be-4f23-b34b-9c97d6080064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:52:31.326390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8f146-64be-4f23-b34b-9c97d6080064 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:52:31.336811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 758/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:52:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:52:33.558034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1475da-570b-44bf-9df7-bf7418e3928e None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:33.558558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1475da-570b-44bf-9df7-bf7418e3928e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:33.558558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1475da-570b-44bf-9df7-bf7418e3928e None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:33.559929 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:52:33.565204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055360b-9615-4bc0-837d-1b6aff472339 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:33.565610 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055360b-9615-4bc0-837d-1b6aff472339 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:33.565758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055360b-9615-4bc0-837d-1b6aff472339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:33.583612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-b055360b-9615-4bc0-837d-1b6aff472339 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:52:33.584793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b055360b-9615-4bc0-837d-1b6aff472339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:33.594513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:52:33.630528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1475da-570b-44bf-9df7-bf7418e3928e None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:52:33.641380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1475da-570b-44bf-9df7-bf7418e3928e None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:52:33.644968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274eb415-ec3d-4f9f-96ec-6f0cbd40a758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:52:33.645604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eb415-ec3d-4f9f-96ec-6f0cbd40a758 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:33.645886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eb415-ec3d-4f9f-96ec-6f0cbd40a758 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:33.646164 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eb415-ec3d-4f9f-96ec-6f0cbd40a758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:33.655657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274eb415-ec3d-4f9f-96ec-6f0cbd40a758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:52:33.657083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274eb415-ec3d-4f9f-96ec-6f0cbd40a758 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:52:33.667184 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 774/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:52:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:52:37.457045 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac803e-3bdf-4612-8d09-4b27427e7e84 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:37.457584 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac803e-3bdf-4612-8d09-4b27427e7e84 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:37.457701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac803e-3bdf-4612-8d09-4b27427e7e84 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:37.459012 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:52:37.464448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3fdd2-522c-44b0-97a0-e0137fa23455 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:37.464889 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3fdd2-522c-44b0-97a0-e0137fa23455 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:37.464987 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3fdd2-522c-44b0-97a0-e0137fa23455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:37.483346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-52c3fdd2-522c-44b0-97a0-e0137fa23455 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:52:37.484582 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-52c3fdd2-522c-44b0-97a0-e0137fa23455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:37.492383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:52:37.764237 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac803e-3bdf-4612-8d09-4b27427e7e84 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:52:37.765925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-89ac803e-3bdf-4612-8d09-4b27427e7e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:37.825632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac803e-3bdf-4612-8d09-4b27427e7e84 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:52:37.829335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8a298-130c-4675-94ab-080a6c916f50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:52:37.829989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8a298-130c-4675-94ab-080a6c916f50 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:37.830275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8a298-130c-4675-94ab-080a6c916f50 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:37.830554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8a298-130c-4675-94ab-080a6c916f50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:37.836537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:52:37.841091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8a298-130c-4675-94ab-080a6c916f50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:52:37.842626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8a298-130c-4675-94ab-080a6c916f50 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:52:37.852007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 761/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:52:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:52:40.118317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcc1de-3f87-4657-96a7-da0f326ad4c5 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:40.119016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcc1de-3f87-4657-96a7-da0f326ad4c5 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:40.119016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcc1de-3f87-4657-96a7-da0f326ad4c5 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:40.121057 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:52:40.126447 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde37d8-66e4-4ca4-b574-0adeac55cb7a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:40.126884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde37d8-66e4-4ca4-b574-0adeac55cb7a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:40.127035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde37d8-66e4-4ca4-b574-0adeac55cb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:40.147705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-7fde37d8-66e4-4ca4-b574-0adeac55cb7a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:52:40.148894 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-7fde37d8-66e4-4ca4-b574-0adeac55cb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:40.156845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:52:40.437185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcc1de-3f87-4657-96a7-da0f326ad4c5 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:52:40.438877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3fbcc1de-3f87-4657-96a7-da0f326ad4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:52:40.446451 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=91 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:52:40.453161 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcc1de-3f87-4657-96a7-da0f326ad4c5 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:52:40.456690 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378633b-ac4d-4182-a21b-e8ab4ba36d16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:52:40.457323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378633b-ac4d-4182-a21b-e8ab4ba36d16 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:52:40.457606 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378633b-ac4d-4182-a21b-e8ab4ba36d16 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:52:40.457882 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378633b-ac4d-4182-a21b-e8ab4ba36d16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:52:40.467681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378633b-ac4d-4182-a21b-e8ab4ba36d16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:52:40.469121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378633b-ac4d-4182-a21b-e8ab4ba36d16 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:52:40.479564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 777/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:52:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:52:50.140506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=6 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:52:50.474346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:10.729512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adcf4e-f98d-4a24-a7fa-c4d44ab3c5d4 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:10.730027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adcf4e-f98d-4a24-a7fa-c4d44ab3c5d4 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:10.730121 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adcf4e-f98d-4a24-a7fa-c4d44ab3c5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:10.764028 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a4adcf4e-f98d-4a24-a7fa-c4d44ab3c5d4 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:10.764549 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ce670-d50a-4bd4-96f8-25c2cbbc138f None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:10.765044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ce670-d50a-4bd4-96f8-25c2cbbc138f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:10.765132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ce670-d50a-4bd4-96f8-25c2cbbc138f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:10.765312 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a4adcf4e-f98d-4a24-a7fa-c4d44ab3c5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:10.782087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 763/1540] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:10.790018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-398ce670-d50a-4bd4-96f8-25c2cbbc138f None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:10.791569 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-398ce670-d50a-4bd4-96f8-25c2cbbc138f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:10.792579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469bc84-c0bf-4fc1-8978-5563efb08d8e None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:10.792881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469bc84-c0bf-4fc1-8978-5563efb08d8e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:10.793151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469bc84-c0bf-4fc1-8978-5563efb08d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:10.812760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 778/1541] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:10.814418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677c77a-0d67-4593-8382-b5312d29b90c None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:10.814966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677c77a-0d67-4593-8382-b5312d29b90c None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:10.815468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677c77a-0d67-4593-8382-b5312d29b90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:10.836195 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-e469bc84-c0bf-4fc1-8978-5563efb08d8e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:10.837910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e469bc84-c0bf-4fc1-8978-5563efb08d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:10.843569 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-a677c77a-0d67-4593-8382-b5312d29b90c None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:10.845146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a677c77a-0d67-4593-8382-b5312d29b90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:10.860407 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 779/1542] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:10.862213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acac7d-f4c3-4fd7-9530-cca3108feb53 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:10.862718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acac7d-f4c3-4fd7-9530-cca3108feb53 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:10.863237 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acac7d-f4c3-4fd7-9530-cca3108feb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:10.864552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 764/1543] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:10.869365 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469bc84-c0bf-4fc1-8978-5563efb08d8e None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:10.888579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469bc84-c0bf-4fc1-8978-5563efb08d8e None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:10.889571 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e1acac7d-f4c3-4fd7-9530-cca3108feb53 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:10.891007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e1acac7d-f4c3-4fd7-9530-cca3108feb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:10.908102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4014a64-1222-4adf-88da-0dfc86df56b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:10.909181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4014a64-1222-4adf-88da-0dfc86df56b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:10.909694 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4014a64-1222-4adf-88da-0dfc86df56b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:10.910185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4014a64-1222-4adf-88da-0dfc86df56b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:10.911186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4014a64-1222-4adf-88da-0dfc86df56b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:10.911703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4014a64-1222-4adf-88da-0dfc86df56b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:10.913692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4014a64-1222-4adf-88da-0dfc86df56b1 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:10.919077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 780/1544] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:10.919077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60eb76-bb5b-4692-9607-1ed9698681c1 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:10.919077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60eb76-bb5b-4692-9607-1ed9698681c1 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:10.919077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60eb76-bb5b-4692-9607-1ed9698681c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:10.933450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 765/1545] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:10.943757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad6451-680b-438c-8052-31a7dd357452 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:10.944428 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad6451-680b-438c-8052-31a7dd357452 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:10.944928 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad6451-680b-438c-8052-31a7dd357452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:10.950944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-5f60eb76-bb5b-4692-9607-1ed9698681c1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:10.952239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5f60eb76-bb5b-4692-9607-1ed9698681c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:10.973144 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 781/1546] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:10.973144 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60eb76-bb5b-4692-9607-1ed9698681c1 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:10.980135 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-eaad6451-680b-438c-8052-31a7dd357452 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:10.981968 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-eaad6451-680b-438c-8052-31a7dd357452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:10.985402 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60eb76-bb5b-4692-9607-1ed9698681c1 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.000713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 766/1547] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:11.000938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058cce97-f2fd-4252-abdd-68a61f16c352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.001625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cce97-f2fd-4252-abdd-68a61f16c352 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.001915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cce97-f2fd-4252-abdd-68a61f16c352 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.002192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cce97-f2fd-4252-abdd-68a61f16c352 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.002730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad6451-680b-438c-8052-31a7dd357452 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.003202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058cce97-f2fd-4252-abdd-68a61f16c352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:11.003513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058cce97-f2fd-4252-abdd-68a61f16c352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.004681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: WARNING py.warnings [None req-058cce97-f2fd-4252-abdd-68a61f16c352 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 13:54:11.004681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: warnings.warn(msg) May 19 13:54:11.004681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: May 19 13:54:11.005603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058cce97-f2fd-4252-abdd-68a61f16c352 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.015386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad6451-680b-438c-8052-31a7dd357452 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.026173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792d727-a053-46ff-bab1-686ce2e426c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.027267 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792d727-a053-46ff-bab1-686ce2e426c8 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.030523 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792d727-a053-46ff-bab1-686ce2e426c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.031093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792d727-a053-46ff-bab1-686ce2e426c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.033543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 782/1548] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:11.034667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792d727-a053-46ff-bab1-686ce2e426c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.035943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: WARNING py.warnings [None req-a792d727-a053-46ff-bab1-686ce2e426c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:54:11.035943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: warnings.warn(msg) May 19 13:54:11.035943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: May 19 13:54:11.037259 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058cce97-f2fd-4252-abdd-68a61f16c352 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.038706 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792d727-a053-46ff-bab1-686ce2e426c8 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.061117 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058cce97-f2fd-4252-abdd-68a61f16c352 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.068005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c0aa0-82ee-4482-aa5e-41e72c6dba46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.069609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c0aa0-82ee-4482-aa5e-41e72c6dba46 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.070125 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c0aa0-82ee-4482-aa5e-41e72c6dba46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.070611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c0aa0-82ee-4482-aa5e-41e72c6dba46 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.071602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c0aa0-82ee-4482-aa5e-41e72c6dba46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:11.072143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c0aa0-82ee-4482-aa5e-41e72c6dba46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.073955 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c0aa0-82ee-4482-aa5e-41e72c6dba46 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.079253 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 767/1549] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/projects => generated 393 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:11.080673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1add0-b258-4302-b1bf-2056ba026841 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.081062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1add0-b258-4302-b1bf-2056ba026841 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.081452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1add0-b258-4302-b1bf-2056ba026841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.103409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-aee1add0-b258-4302-b1bf-2056ba026841 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:11.104657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-aee1add0-b258-4302-b1bf-2056ba026841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:11.177608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 768/1550] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:11.179135 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1add0-b258-4302-b1bf-2056ba026841 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.190586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1add0-b258-4302-b1bf-2056ba026841 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.195442 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdfac4-a5f9-47c8-a0a0-f74cff588417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.196115 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdfac4-a5f9-47c8-a0a0-f74cff588417 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.196397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdfac4-a5f9-47c8-a0a0-f74cff588417 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.196668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdfac4-a5f9-47c8-a0a0-f74cff588417 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.197360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdfac4-a5f9-47c8-a0a0-f74cff588417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.199286 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdfac4-a5f9-47c8-a0a0-f74cff588417 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.220871 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 783/1551] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:11.230009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 769/1552] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:11.230009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad6127-b8b7-48a9-8497-5bce44c899cc None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.230009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad6127-b8b7-48a9-8497-5bce44c899cc None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.230328 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024609d-a0e1-406a-a860-8de609369671 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.230426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad6127-b8b7-48a9-8497-5bce44c899cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.230514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024609d-a0e1-406a-a860-8de609369671 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.230514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024609d-a0e1-406a-a860-8de609369671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.248915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-b024609d-a0e1-406a-a860-8de609369671 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:11.249136 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-43ad6127-b8b7-48a9-8497-5bce44c899cc None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:11.250117 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-b024609d-a0e1-406a-a860-8de609369671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:11.250247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-43ad6127-b8b7-48a9-8497-5bce44c899cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:11.308136 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 770/1553] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:11.309854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad6127-b8b7-48a9-8497-5bce44c899cc None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.319130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 784/1554] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:11.319130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99338a34-9256-41ab-ab8a-100fcf5d04eb None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.319130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99338a34-9256-41ab-ab8a-100fcf5d04eb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.319130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99338a34-9256-41ab-ab8a-100fcf5d04eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.319601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad6127-b8b7-48a9-8497-5bce44c899cc None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.324076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a57e7-698e-4297-9dcb-99c3e9aa0d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.324810 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a57e7-698e-4297-9dcb-99c3e9aa0d6a admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.325090 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a57e7-698e-4297-9dcb-99c3e9aa0d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.325365 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a57e7-698e-4297-9dcb-99c3e9aa0d6a admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.326055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a57e7-698e-4297-9dcb-99c3e9aa0d6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.327994 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a57e7-698e-4297-9dcb-99c3e9aa0d6a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.336270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-99338a34-9256-41ab-ab8a-100fcf5d04eb None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:11.337466 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-99338a34-9256-41ab-ab8a-100fcf5d04eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:11.346167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 785/1555] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:11.347257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8b74a-c4af-41db-99ba-d7007aaff012 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.347544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8b74a-c4af-41db-99ba-d7007aaff012 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.347819 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8b74a-c4af-41db-99ba-d7007aaff012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.351405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 771/1556] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/users => generated 297 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:11.351405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a57e7-698e-4297-9dcb-99c3e9aa0d6a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.362063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a57e7-698e-4297-9dcb-99c3e9aa0d6a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.369827 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b158172-d034-4120-9dea-38b51885c69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.370788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b158172-d034-4120-9dea-38b51885c69d admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.371205 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b158172-d034-4120-9dea-38b51885c69d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.372242 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b158172-d034-4120-9dea-38b51885c69d admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.372969 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b158172-d034-4120-9dea-38b51885c69d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:11.373275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b158172-d034-4120-9dea-38b51885c69d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.374371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f5e8b74a-c4af-41db-99ba-d7007aaff012 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:11.374882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b158172-d034-4120-9dea-38b51885c69d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.375591 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f5e8b74a-c4af-41db-99ba-d7007aaff012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:11.376764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:11.384179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 786/1557] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:11.384745 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8b74a-c4af-41db-99ba-d7007aaff012 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.390938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 772/1558] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:11.398272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b158172-d034-4120-9dea-38b51885c69d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.404473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8b74a-c4af-41db-99ba-d7007aaff012 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.406150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b158172-d034-4120-9dea-38b51885c69d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.409278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99ff0d-4edf-4e4d-8083-6a706aee45ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.409932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99ff0d-4edf-4e4d-8083-6a706aee45ef admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.410214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99ff0d-4edf-4e4d-8083-6a706aee45ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.410504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99ff0d-4edf-4e4d-8083-6a706aee45ef admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.410726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b81d17-2fbd-499a-81df-ae2937f8e717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.411215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99ff0d-4edf-4e4d-8083-6a706aee45ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.411456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b81d17-2fbd-499a-81df-ae2937f8e717 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.411744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b81d17-2fbd-499a-81df-ae2937f8e717 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.412020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b81d17-2fbd-499a-81df-ae2937f8e717 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.412700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b81d17-2fbd-499a-81df-ae2937f8e717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.413091 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99ff0d-4edf-4e4d-8083-6a706aee45ef admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.414603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b81d17-2fbd-499a-81df-ae2937f8e717 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.436461 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 773/1559] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:11.437730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b29d46-ec39-43a9-b342-8b9d59d8d2f8 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.438020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b29d46-ec39-43a9-b342-8b9d59d8d2f8 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.438298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b29d46-ec39-43a9-b342-8b9d59d8d2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.444593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 787/1560] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:11.444593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019d1a6-3e40-44f3-a6cd-e1f33ff2fea5 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.444593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019d1a6-3e40-44f3-a6cd-e1f33ff2fea5 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.444593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019d1a6-3e40-44f3-a6cd-e1f33ff2fea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.456795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-b1b29d46-ec39-43a9-b342-8b9d59d8d2f8 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:11.458095 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b1b29d46-ec39-43a9-b342-8b9d59d8d2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:11.461553 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-7019d1a6-3e40-44f3-a6cd-e1f33ff2fea5 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:11.462759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-7019d1a6-3e40-44f3-a6cd-e1f33ff2fea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:11.520785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 774/1561] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:11.521545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b29d46-ec39-43a9-b342-8b9d59d8d2f8 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.532002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 788/1562] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:11.532002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a38ca-84ad-42fd-90f7-d1cac796ad72 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.532002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a38ca-84ad-42fd-90f7-d1cac796ad72 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.532002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a38ca-84ad-42fd-90f7-d1cac796ad72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.532476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b29d46-ec39-43a9-b342-8b9d59d8d2f8 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.536567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cf062-a34c-4fb8-bc40-125321164c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.537222 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cf062-a34c-4fb8-bc40-125321164c2c admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.537504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cf062-a34c-4fb8-bc40-125321164c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.537779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cf062-a34c-4fb8-bc40-125321164c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.538508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cf062-a34c-4fb8-bc40-125321164c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.540133 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cf062-a34c-4fb8-bc40-125321164c2c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.548468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-545a38ca-84ad-42fd-90f7-d1cac796ad72 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:11.549669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-545a38ca-84ad-42fd-90f7-d1cac796ad72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:11.550517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 775/1563] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:11.551704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1957d7-1ebb-46a6-9213-12eb83eaf32f None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.551994 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1957d7-1ebb-46a6-9213-12eb83eaf32f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.552273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1957d7-1ebb-46a6-9213-12eb83eaf32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.559935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 789/1564] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:11.560904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a38ca-84ad-42fd-90f7-d1cac796ad72 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.570553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6b1957d7-1ebb-46a6-9213-12eb83eaf32f None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:11.570729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a38ca-84ad-42fd-90f7-d1cac796ad72 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.571771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6b1957d7-1ebb-46a6-9213-12eb83eaf32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:11.576016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ac749-588a-4788-b579-f398a0d9b9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.576676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ac749-588a-4788-b579-f398a0d9b9ee admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.576975 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ac749-588a-4788-b579-f398a0d9b9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.577280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ac749-588a-4788-b579-f398a0d9b9ee admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.578042 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ac749-588a-4788-b579-f398a0d9b9ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.580151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ac749-588a-4788-b579-f398a0d9b9ee admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.584159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 776/1565] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:11.584159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1957d7-1ebb-46a6-9213-12eb83eaf32f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.584407 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:11.591535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1957d7-1ebb-46a6-9213-12eb83eaf32f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.596255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9affcd0-040c-4952-9a70-a87d7647b154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.596910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9affcd0-040c-4952-9a70-a87d7647b154 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.597251 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9affcd0-040c-4952-9a70-a87d7647b154 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.597530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9affcd0-040c-4952-9a70-a87d7647b154 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.598250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9affcd0-040c-4952-9a70-a87d7647b154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.599867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9affcd0-040c-4952-9a70-a87d7647b154 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.610666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 777/1566] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:11.610666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9affcd0-040c-4952-9a70-a87d7647b154 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.615472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 790/1567] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:11.616514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ac749-588a-4788-b579-f398a0d9b9ee admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.623396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9affcd0-040c-4952-9a70-a87d7647b154 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.679740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ac749-588a-4788-b579-f398a0d9b9ee admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.680429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e137d8e-de08-442c-83ea-fcd0fa93cd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.681106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e137d8e-de08-442c-83ea-fcd0fa93cd68 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.681389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e137d8e-de08-442c-83ea-fcd0fa93cd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.681663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e137d8e-de08-442c-83ea-fcd0fa93cd68 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.682344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e137d8e-de08-442c-83ea-fcd0fa93cd68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.684562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4100f45-26ab-4fa5-8a90-b597cd332ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.684679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e137d8e-de08-442c-83ea-fcd0fa93cd68 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.685226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4100f45-26ab-4fa5-8a90-b597cd332ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.685509 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4100f45-26ab-4fa5-8a90-b597cd332ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.685785 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4100f45-26ab-4fa5-8a90-b597cd332ef6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.686509 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4100f45-26ab-4fa5-8a90-b597cd332ef6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:11.686814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4100f45-26ab-4fa5-8a90-b597cd332ef6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.688417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4100f45-26ab-4fa5-8a90-b597cd332ef6 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.702629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 791/1568] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:11.704421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4100f45-26ab-4fa5-8a90-b597cd332ef6 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.717263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 778/1569] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:11.718096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e137d8e-de08-442c-83ea-fcd0fa93cd68 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.720427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4100f45-26ab-4fa5-8a90-b597cd332ef6 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.725808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353164e-ad79-4ac5-bada-184298af9d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.726730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353164e-ad79-4ac5-bada-184298af9d35 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.727137 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353164e-ad79-4ac5-bada-184298af9d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.727471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353164e-ad79-4ac5-bada-184298af9d35 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.728210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e137d8e-de08-442c-83ea-fcd0fa93cd68 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.728317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353164e-ad79-4ac5-bada-184298af9d35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:11.728728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353164e-ad79-4ac5-bada-184298af9d35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.730896 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353164e-ad79-4ac5-bada-184298af9d35 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.733793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bcdb2-f681-4985-88a0-ad135830388a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.734617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bcdb2-f681-4985-88a0-ad135830388a admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.734759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bcdb2-f681-4985-88a0-ad135830388a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.734985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bcdb2-f681-4985-88a0-ad135830388a admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.735706 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bcdb2-f681-4985-88a0-ad135830388a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.737291 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bcdb2-f681-4985-88a0-ad135830388a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.752320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 792/1570] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:11.752320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353164e-ad79-4ac5-bada-184298af9d35 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.752607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 779/1571] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:11.752607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bcdb2-f681-4985-88a0-ad135830388a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.756927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353164e-ad79-4ac5-bada-184298af9d35 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.760098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bcdb2-f681-4985-88a0-ad135830388a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.761504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad708a98-c2a3-409c-b2dc-fab5b8e75157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.762202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad708a98-c2a3-409c-b2dc-fab5b8e75157 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.762482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad708a98-c2a3-409c-b2dc-fab5b8e75157 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.762761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad708a98-c2a3-409c-b2dc-fab5b8e75157 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.763511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad708a98-c2a3-409c-b2dc-fab5b8e75157 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:11.763844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad708a98-c2a3-409c-b2dc-fab5b8e75157 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.765198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b23dd7-72b8-412a-9e8a-01aaab5e3615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.765415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad708a98-c2a3-409c-b2dc-fab5b8e75157 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.765891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b23dd7-72b8-412a-9e8a-01aaab5e3615 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.766106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b23dd7-72b8-412a-9e8a-01aaab5e3615 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.766382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b23dd7-72b8-412a-9e8a-01aaab5e3615 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.767206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b23dd7-72b8-412a-9e8a-01aaab5e3615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:11.767514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b23dd7-72b8-412a-9e8a-01aaab5e3615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.769073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b23dd7-72b8-412a-9e8a-01aaab5e3615 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.778968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 793/1572] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:11.789304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 780/1573] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:11.789304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b23dd7-72b8-412a-9e8a-01aaab5e3615 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.789587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad708a98-c2a3-409c-b2dc-fab5b8e75157 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.794350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad708a98-c2a3-409c-b2dc-fab5b8e75157 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.795932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b23dd7-72b8-412a-9e8a-01aaab5e3615 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.798943 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992d34a-6f05-4a48-bd62-b0328d69f5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.799613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992d34a-6f05-4a48-bd62-b0328d69f5ee admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.799894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992d34a-6f05-4a48-bd62-b0328d69f5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.800171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992d34a-6f05-4a48-bd62-b0328d69f5ee admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.800729 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77b20e-e867-4f76-b328-568e7c7a9b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.800890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992d34a-6f05-4a48-bd62-b0328d69f5ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.801382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77b20e-e867-4f76-b328-568e7c7a9b06 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.801666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77b20e-e867-4f76-b328-568e7c7a9b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.801947 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77b20e-e867-4f76-b328-568e7c7a9b06 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.802481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992d34a-6f05-4a48-bd62-b0328d69f5ee admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.802624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e77b20e-e867-4f76-b328-568e7c7a9b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.805111 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e77b20e-e867-4f76-b328-568e7c7a9b06 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.812408 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 794/1574] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:11.813176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992d34a-6f05-4a48-bd62-b0328d69f5ee admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.818633 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:11.823477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992d34a-6f05-4a48-bd62-b0328d69f5ee admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.829143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccab95-6727-484d-b21d-9fd2e8198ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.829813 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccab95-6727-484d-b21d-9fd2e8198ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.830093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccab95-6727-484d-b21d-9fd2e8198ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.830364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccab95-6727-484d-b21d-9fd2e8198ce4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.833956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 781/1575] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/users => generated 288 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:11.834101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccab95-6727-484d-b21d-9fd2e8198ce4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.834209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77b20e-e867-4f76-b328-568e7c7a9b06 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.834310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccab95-6727-484d-b21d-9fd2e8198ce4 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.843114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77b20e-e867-4f76-b328-568e7c7a9b06 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.847786 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba23a47-96cb-4443-90c9-ad6bbbba2e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.848440 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba23a47-96cb-4443-90c9-ad6bbbba2e7c admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.848734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba23a47-96cb-4443-90c9-ad6bbbba2e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.849005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba23a47-96cb-4443-90c9-ad6bbbba2e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.849754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba23a47-96cb-4443-90c9-ad6bbbba2e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.851416 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba23a47-96cb-4443-90c9-ad6bbbba2e7c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.859624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 795/1576] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:11.860610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccab95-6727-484d-b21d-9fd2e8198ce4 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.862973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 782/1577] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:11.863839 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba23a47-96cb-4443-90c9-ad6bbbba2e7c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.873958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba23a47-96cb-4443-90c9-ad6bbbba2e7c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.923945 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccab95-6727-484d-b21d-9fd2e8198ce4 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:11.928534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac32bd5-4113-4e40-8fba-d6ae4eb06be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.929441 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac32bd5-4113-4e40-8fba-d6ae4eb06be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.929723 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac32bd5-4113-4e40-8fba-d6ae4eb06be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.929997 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac32bd5-4113-4e40-8fba-d6ae4eb06be9 admin admin] PATH_INFO: `/v3/projects/28027af4cc0c4ca99035e48aa6237805/users/698ccea06fb24beea18c679d53f907a6/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.931042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ed245-b8b9-4eea-b6a2-396cbe949b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:11.931862 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ed245-b8b9-4eea-b6a2-396cbe949b90 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:11.932150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ed245-b8b9-4eea-b6a2-396cbe949b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:11.932428 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ed245-b8b9-4eea-b6a2-396cbe949b90 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:11.933114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ed245-b8b9-4eea-b6a2-396cbe949b90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.935329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ed245-b8b9-4eea-b6a2-396cbe949b90 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.944763 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac32bd5-4113-4e40-8fba-d6ae4eb06be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28027af4cc0c4ca99035e48aa6237805, user_id=698ccea06fb24beea18c679d53f907a6, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:11.947420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac32bd5-4113-4e40-8fba-d6ae4eb06be9 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:11.961123 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 783/1578] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:11.961682 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 796/1579] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:11 2022] PUT /identity/v3/projects/28027af4cc0c4ca99035e48aa6237805/users/698ccea06fb24beea18c679d53f907a6/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:11.962376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac32bd5-4113-4e40-8fba-d6ae4eb06be9 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.965086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ed245-b8b9-4eea-b6a2-396cbe949b90 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:11.967131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:12.027937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ed245-b8b9-4eea-b6a2-396cbe949b90 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.034342 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697d3f7-1a8f-4b91-b190-ed67be396a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.034986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697d3f7-1a8f-4b91-b190-ed67be396a24 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.035127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac32bd5-4113-4e40-8fba-d6ae4eb06be9 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.035307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697d3f7-1a8f-4b91-b190-ed67be396a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.035558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697d3f7-1a8f-4b91-b190-ed67be396a24 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.036252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697d3f7-1a8f-4b91-b190-ed67be396a24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.038165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697d3f7-1a8f-4b91-b190-ed67be396a24 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.039788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89d203-42ec-44af-9de6-315b78748033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.040436 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89d203-42ec-44af-9de6-315b78748033 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.040719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89d203-42ec-44af-9de6-315b78748033 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.040999 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89d203-42ec-44af-9de6-315b78748033 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.041720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89d203-42ec-44af-9de6-315b78748033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.044146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89d203-42ec-44af-9de6-315b78748033 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.066306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 784/1580] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:12.067162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697d3f7-1a8f-4b91-b190-ed67be396a24 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.071521 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 797/1581] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:11 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:12.072510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89d203-42ec-44af-9de6-315b78748033 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.128300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:12.133154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697d3f7-1a8f-4b91-b190-ed67be396a24 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.136710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89d203-42ec-44af-9de6-315b78748033 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.138758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785f49e-5786-4ae2-bb77-66617c8b20d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.139692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785f49e-5786-4ae2-bb77-66617c8b20d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.140025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785f49e-5786-4ae2-bb77-66617c8b20d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.140258 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785f49e-5786-4ae2-bb77-66617c8b20d9 admin admin] PATH_INFO: `/v3/projects/0b1f5c694f5f4ce1b8620ec6d85d8861/users/947e6f420c0d490699fb047c4e4a6dad/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.141580 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7325f6f-4614-4c23-981b-645601182775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.142242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7325f6f-4614-4c23-981b-645601182775 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.142527 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7325f6f-4614-4c23-981b-645601182775 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.142950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7325f6f-4614-4c23-981b-645601182775 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.143680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7325f6f-4614-4c23-981b-645601182775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.145261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7325f6f-4614-4c23-981b-645601182775 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.155473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 798/1582] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:12.155727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785f49e-5786-4ae2-bb77-66617c8b20d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1f5c694f5f4ce1b8620ec6d85d8861, user_id=947e6f420c0d490699fb047c4e4a6dad, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.156259 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7325f6f-4614-4c23-981b-645601182775 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.156945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: WARNING py.warnings [None req-2785f49e-5786-4ae2-bb77-66617c8b20d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:54:12.156945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: warnings.warn(msg) May 19 13:54:12.156945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: May 19 13:54:12.158945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785f49e-5786-4ae2-bb77-66617c8b20d9 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.167961 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7325f6f-4614-4c23-981b-645601182775 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.174376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 785/1583] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:12 2022] PUT /identity/v3/projects/0b1f5c694f5f4ce1b8620ec6d85d8861/users/947e6f420c0d490699fb047c4e4a6dad/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:12.179862 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785f49e-5786-4ae2-bb77-66617c8b20d9 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.227041 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81b017-aa29-45df-9120-957e96cd3745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.227716 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81b017-aa29-45df-9120-957e96cd3745 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.228000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81b017-aa29-45df-9120-957e96cd3745 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.228306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81b017-aa29-45df-9120-957e96cd3745 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.229023 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81b017-aa29-45df-9120-957e96cd3745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.230068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: WARNING py.warnings [None req-bb81b017-aa29-45df-9120-957e96cd3745 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:54:12.230068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: warnings.warn(msg) May 19 13:54:12.230068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: May 19 13:54:12.231547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81b017-aa29-45df-9120-957e96cd3745 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.244309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785f49e-5786-4ae2-bb77-66617c8b20d9 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.249705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3650e-16cc-4ecd-aa29-5ea7dd6b7096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.250371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3650e-16cc-4ecd-aa29-5ea7dd6b7096 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.250659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3650e-16cc-4ecd-aa29-5ea7dd6b7096 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.250969 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3650e-16cc-4ecd-aa29-5ea7dd6b7096 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.251700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3650e-16cc-4ecd-aa29-5ea7dd6b7096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.252031 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 799/1584] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:12 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:12.253080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81b017-aa29-45df-9120-957e96cd3745 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.253652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3650e-16cc-4ecd-aa29-5ea7dd6b7096 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.268719 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 786/1585] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:12.268947 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81b017-aa29-45df-9120-957e96cd3745 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.269049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667c617-3a3c-47d2-9c83-93e9d79231bf None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.269049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667c617-3a3c-47d2-9c83-93e9d79231bf None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.269049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667c617-3a3c-47d2-9c83-93e9d79231bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.270336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e68017-98b4-4969-8f5c-4933512bde25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.271025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e68017-98b4-4969-8f5c-4933512bde25 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.271368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e68017-98b4-4969-8f5c-4933512bde25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.271650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e68017-98b4-4969-8f5c-4933512bde25 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.272340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e68017-98b4-4969-8f5c-4933512bde25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.274277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e68017-98b4-4969-8f5c-4933512bde25 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.284319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:12.285712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6667c617-3a3c-47d2-9c83-93e9d79231bf None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:12.286906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6667c617-3a3c-47d2-9c83-93e9d79231bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:12.295074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 787/1586] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:12.295833 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667c617-3a3c-47d2-9c83-93e9d79231bf None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.302495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 800/1587] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:12 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:12.302495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e68017-98b4-4969-8f5c-4933512bde25 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.305264 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667c617-3a3c-47d2-9c83-93e9d79231bf None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.310741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e49ce-5b10-4506-8c50-06574f1b1314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.311484 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e49ce-5b10-4506-8c50-06574f1b1314 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.311803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e49ce-5b10-4506-8c50-06574f1b1314 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.312080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e49ce-5b10-4506-8c50-06574f1b1314 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.312448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e68017-98b4-4969-8f5c-4933512bde25 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.312764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e49ce-5b10-4506-8c50-06574f1b1314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.314703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e49ce-5b10-4506-8c50-06574f1b1314 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.317043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5392919-2187-4837-a6eb-bc52bbe8cb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.317687 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5392919-2187-4837-a6eb-bc52bbe8cb38 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.317970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5392919-2187-4837-a6eb-bc52bbe8cb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.318245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5392919-2187-4837-a6eb-bc52bbe8cb38 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.318961 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5392919-2187-4837-a6eb-bc52bbe8cb38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.320889 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5392919-2187-4837-a6eb-bc52bbe8cb38 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.335768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 788/1588] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:12 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:12.336785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e49ce-5b10-4506-8c50-06574f1b1314 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.346419 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 801/1589] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:12 2022] POST /identity/v3/users => generated 295 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:12.347579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1630ae-6d22-4fb2-b25b-bda2101a1507 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.347875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1630ae-6d22-4fb2-b25b-bda2101a1507 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.348060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e49ce-5b10-4506-8c50-06574f1b1314 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.348161 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1630ae-6d22-4fb2-b25b-bda2101a1507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.352687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71090605-0350-49f4-a65b-52ce0149ead1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.353337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71090605-0350-49f4-a65b-52ce0149ead1 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.353622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71090605-0350-49f4-a65b-52ce0149ead1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.353903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71090605-0350-49f4-a65b-52ce0149ead1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.354626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71090605-0350-49f4-a65b-52ce0149ead1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.356826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71090605-0350-49f4-a65b-52ce0149ead1 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.366901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 789/1590] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:12.370891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71090605-0350-49f4-a65b-52ce0149ead1 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.374801 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-af1630ae-6d22-4fb2-b25b-bda2101a1507 None None] MFA Rules not processed for user `947e6f420c0d490699fb047c4e4a6dad`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:12.376073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-af1630ae-6d22-4fb2-b25b-bda2101a1507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:12.381063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71090605-0350-49f4-a65b-52ce0149ead1 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.385639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d2754-9762-4027-bd18-ae53aafe0112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.386287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d2754-9762-4027-bd18-ae53aafe0112 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.386569 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d2754-9762-4027-bd18-ae53aafe0112 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.386922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d2754-9762-4027-bd18-ae53aafe0112 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.387804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d2754-9762-4027-bd18-ae53aafe0112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.389380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d2754-9762-4027-bd18-ae53aafe0112 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.400341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 790/1591] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:12.401142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d2754-9762-4027-bd18-ae53aafe0112 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.411058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d2754-9762-4027-bd18-ae53aafe0112 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.415843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172358e2-7bde-4f34-be17-e896212693bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.417345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172358e2-7bde-4f34-be17-e896212693bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.417628 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172358e2-7bde-4f34-be17-e896212693bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.417901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172358e2-7bde-4f34-be17-e896212693bb admin admin] PATH_INFO: `/v3/projects/c58c574bbe6042f9be50dd29194adfa3/users/20f19aa4d350436a98c0bf36d1973143/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.434540 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172358e2-7bde-4f34-be17-e896212693bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58c574bbe6042f9be50dd29194adfa3, user_id=20f19aa4d350436a98c0bf36d1973143, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.437219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172358e2-7bde-4f34-be17-e896212693bb admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.448622 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 802/1592] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:12.449473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1630ae-6d22-4fb2-b25b-bda2101a1507 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.459174 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 791/1593] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:12 2022] PUT /identity/v3/projects/c58c574bbe6042f9be50dd29194adfa3/users/20f19aa4d350436a98c0bf36d1973143/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:12.459174 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172358e2-7bde-4f34-be17-e896212693bb admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.469650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1630ae-6d22-4fb2-b25b-bda2101a1507 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.510825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172358e2-7bde-4f34-be17-e896212693bb admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.515281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22155a-36bc-4fd9-a3d1-a0050fc0bc12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.515932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22155a-36bc-4fd9-a3d1-a0050fc0bc12 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.516216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22155a-36bc-4fd9-a3d1-a0050fc0bc12 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.516493 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22155a-36bc-4fd9-a3d1-a0050fc0bc12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.525508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbb1ce-1d6f-4683-a24e-20d9153a530b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.526203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbb1ce-1d6f-4683-a24e-20d9153a530b admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.526510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbb1ce-1d6f-4683-a24e-20d9153a530b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.526783 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbb1ce-1d6f-4683-a24e-20d9153a530b admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.527538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbb1ce-1d6f-4683-a24e-20d9153a530b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.529542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbb1ce-1d6f-4683-a24e-20d9153a530b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.534675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:12.540692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 803/1594] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:12.541496 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbb1ce-1d6f-4683-a24e-20d9153a530b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.551747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbb1ce-1d6f-4683-a24e-20d9153a530b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.556277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b3732-ce0a-4a48-b065-c140aa447540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.556921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b3732-ce0a-4a48-b065-c140aa447540 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.557203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b3732-ce0a-4a48-b065-c140aa447540 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.557477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b3732-ce0a-4a48-b065-c140aa447540 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.558195 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b3732-ce0a-4a48-b065-c140aa447540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.560353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b3732-ce0a-4a48-b065-c140aa447540 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.570875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 804/1595] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:12.571711 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b3732-ce0a-4a48-b065-c140aa447540 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.581857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b3732-ce0a-4a48-b065-c140aa447540 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.585252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22155a-36bc-4fd9-a3d1-a0050fc0bc12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.586467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb4e0b-8959-427b-bf09-a321c59b3e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.586713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22155a-36bc-4fd9-a3d1-a0050fc0bc12 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.587121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb4e0b-8959-427b-bf09-a321c59b3e57 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.587405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb4e0b-8959-427b-bf09-a321c59b3e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.587679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb4e0b-8959-427b-bf09-a321c59b3e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.588382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb4e0b-8959-427b-bf09-a321c59b3e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.589919 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb4e0b-8959-427b-bf09-a321c59b3e57 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.596805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 792/1596] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:12.597469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22155a-36bc-4fd9-a3d1-a0050fc0bc12 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.603091 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 805/1597] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:12.603091 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb4e0b-8959-427b-bf09-a321c59b3e57 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.609139 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22155a-36bc-4fd9-a3d1-a0050fc0bc12 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.610951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb4e0b-8959-427b-bf09-a321c59b3e57 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.614536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be3f5e-9db3-4a8d-bf70-7e65c2ad6ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.615458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be3f5e-9db3-4a8d-bf70-7e65c2ad6ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.615595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c4d3b-c849-4c32-935a-4914f494ef6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.615724 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be3f5e-9db3-4a8d-bf70-7e65c2ad6ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.616002 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be3f5e-9db3-4a8d-bf70-7e65c2ad6ed0 admin admin] PATH_INFO: `/v3/projects/d5a0c47cbf434c64a59b1ce21027ce70/users/264a78e35a654cfea772c2a1d4f8a0d9/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.616235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c4d3b-c849-4c32-935a-4914f494ef6a admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.616517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c4d3b-c849-4c32-935a-4914f494ef6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.616788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c4d3b-c849-4c32-935a-4914f494ef6a admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.617504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c4d3b-c849-4c32-935a-4914f494ef6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.619189 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c4d3b-c849-4c32-935a-4914f494ef6a admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.630625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 806/1598] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:12.634312 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be3f5e-9db3-4a8d-bf70-7e65c2ad6ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a0c47cbf434c64a59b1ce21027ce70, user_id=264a78e35a654cfea772c2a1d4f8a0d9, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.634494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c4d3b-c849-4c32-935a-4914f494ef6a admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.636999 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be3f5e-9db3-4a8d-bf70-7e65c2ad6ed0 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.645377 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c4d3b-c849-4c32-935a-4914f494ef6a admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.651945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 793/1599] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:12 2022] PUT /identity/v3/projects/d5a0c47cbf434c64a59b1ce21027ce70/users/264a78e35a654cfea772c2a1d4f8a0d9/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:12.654089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc52e2-d172-48b8-b5d5-88ca40d6dd1b None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.654089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc52e2-d172-48b8-b5d5-88ca40d6dd1b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.655024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc52e2-d172-48b8-b5d5-88ca40d6dd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.691217 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:12.693871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-98cc52e2-d172-48b8-b5d5-88ca40d6dd1b None None] MFA Rules not processed for user `20f19aa4d350436a98c0bf36d1973143`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:12.696351 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-98cc52e2-d172-48b8-b5d5-88ca40d6dd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:12.726260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba8e62-04ed-4c01-ab29-0b9e9f1ea25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.727124 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba8e62-04ed-4c01-ab29-0b9e9f1ea25f admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.727349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba8e62-04ed-4c01-ab29-0b9e9f1ea25f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.727598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba8e62-04ed-4c01-ab29-0b9e9f1ea25f admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.728353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba8e62-04ed-4c01-ab29-0b9e9f1ea25f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.730440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba8e62-04ed-4c01-ab29-0b9e9f1ea25f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.764357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 807/1600] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:12 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:12.765279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba8e62-04ed-4c01-ab29-0b9e9f1ea25f admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.769858 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 794/1601] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:12.770784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc52e2-d172-48b8-b5d5-88ca40d6dd1b None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.784593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc52e2-d172-48b8-b5d5-88ca40d6dd1b None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.847898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3a456-e2c1-428e-8f76-2e39fa5c6afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.848589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3a456-e2c1-428e-8f76-2e39fa5c6afb admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.848877 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3a456-e2c1-428e-8f76-2e39fa5c6afb admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.849159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3a456-e2c1-428e-8f76-2e39fa5c6afb admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.849893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3a456-e2c1-428e-8f76-2e39fa5c6afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.851764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3a456-e2c1-428e-8f76-2e39fa5c6afb admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.862336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 795/1602] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:12.863107 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3a456-e2c1-428e-8f76-2e39fa5c6afb admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.865216 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba8e62-04ed-4c01-ab29-0b9e9f1ea25f admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.872945 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1bbf3-5b65-41e3-af77-778182e64b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.873388 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3a456-e2c1-428e-8f76-2e39fa5c6afb admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.873887 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1bbf3-5b65-41e3-af77-778182e64b14 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.874321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1bbf3-5b65-41e3-af77-778182e64b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.874678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1bbf3-5b65-41e3-af77-778182e64b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.875535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1bbf3-5b65-41e3-af77-778182e64b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.878449 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1bbf3-5b65-41e3-af77-778182e64b14 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.878714 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be8b47-c26d-45dd-980b-653825bec479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.879608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be8b47-c26d-45dd-980b-653825bec479 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.879736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be8b47-c26d-45dd-980b-653825bec479 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.879964 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be8b47-c26d-45dd-980b-653825bec479 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.880692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be8b47-c26d-45dd-980b-653825bec479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.882279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be8b47-c26d-45dd-980b-653825bec479 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.890183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 808/1603] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:12.891031 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1bbf3-5b65-41e3-af77-778182e64b14 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:12.896853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 796/1604] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:12.898456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d09c-d335-494d-b659-df0c505ad9a5 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.898764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d09c-d335-494d-b659-df0c505ad9a5 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.899086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d09c-d335-494d-b659-df0c505ad9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.907423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1bbf3-5b65-41e3-af77-778182e64b14 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:12.907775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:12.913621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75b106-8d1b-46fa-bc52-8dfd33709199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:12.914668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75b106-8d1b-46fa-bc52-8dfd33709199 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:12.915096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75b106-8d1b-46fa-bc52-8dfd33709199 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:12.915481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75b106-8d1b-46fa-bc52-8dfd33709199 admin admin] PATH_INFO: `/v3/projects/89b4336043d5446b922432a2f43af17a/users/0cc3e0105f2a472092423852b1f39979/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:12.935586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-5d25d09c-d335-494d-b659-df0c505ad9a5 None None] MFA Rules not processed for user `264a78e35a654cfea772c2a1d4f8a0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:12.937187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5d25d09c-d335-494d-b659-df0c505ad9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:12.939728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75b106-8d1b-46fa-bc52-8dfd33709199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b4336043d5446b922432a2f43af17a, user_id=0cc3e0105f2a472092423852b1f39979, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:12.942763 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75b106-8d1b-46fa-bc52-8dfd33709199 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:12.964668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 809/1605] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:12 2022] PUT /identity/v3/projects/89b4336043d5446b922432a2f43af17a/users/0cc3e0105f2a472092423852b1f39979/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:12.966805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75b106-8d1b-46fa-bc52-8dfd33709199 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:13.028904 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 797/1606] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:13.029542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d09c-d335-494d-b659-df0c505ad9a5 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:13.045206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d09c-d335-494d-b659-df0c505ad9a5 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:13.051150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0bf28-8aab-4293-bbe6-543549a0d641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:13.052543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0bf28-8aab-4293-bbe6-543549a0d641 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.054189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0bf28-8aab-4293-bbe6-543549a0d641 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.054780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0bf28-8aab-4293-bbe6-543549a0d641 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.055363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0bf28-8aab-4293-bbe6-543549a0d641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:13.057381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0bf28-8aab-4293-bbe6-543549a0d641 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:13.057505 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75b106-8d1b-46fa-bc52-8dfd33709199 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:13.066190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450fe60-b792-4998-bd4b-02dcab020aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:13.066905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450fe60-b792-4998-bd4b-02dcab020aed admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.067349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450fe60-b792-4998-bd4b-02dcab020aed admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.067641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450fe60-b792-4998-bd4b-02dcab020aed admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.068502 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450fe60-b792-4998-bd4b-02dcab020aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:13.071436 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450fe60-b792-4998-bd4b-02dcab020aed admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:13.092396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 798/1607] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:13 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:13.093221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0bf28-8aab-4293-bbe6-543549a0d641 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:13.101215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 810/1608] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:12 2022] POST /identity/v3/users => generated 297 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:13.101215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450fe60-b792-4998-bd4b-02dcab020aed admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:13.113364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450fe60-b792-4998-bd4b-02dcab020aed admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:13.129170 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:13.164757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0bf28-8aab-4293-bbe6-543549a0d641 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:13.171313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936ceb8-eb55-43f1-af8a-d2cebe678742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:13.171950 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936ceb8-eb55-43f1-af8a-d2cebe678742 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.172231 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936ceb8-eb55-43f1-af8a-d2cebe678742 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.172509 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936ceb8-eb55-43f1-af8a-d2cebe678742 admin admin] PATH_INFO: `/v3/projects/4ea632c21d7549c89749672a6f65996d/users/c3b964fb4e024354ba066b38537d2da9/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.188049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400ccb4-23a4-48e9-88a3-b775c1b7806c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:13.188958 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400ccb4-23a4-48e9-88a3-b775c1b7806c admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.189202 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936ceb8-eb55-43f1-af8a-d2cebe678742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea632c21d7549c89749672a6f65996d, user_id=c3b964fb4e024354ba066b38537d2da9, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:13.189321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400ccb4-23a4-48e9-88a3-b775c1b7806c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.189570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400ccb4-23a4-48e9-88a3-b775c1b7806c admin admin] PATH_INFO: `/v3/projects/89676dbc35634ec7994040c93f693f7c/users/fd88c8e9166940d8a09657997e6214d6/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.192060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936ceb8-eb55-43f1-af8a-d2cebe678742 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:13.205238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400ccb4-23a4-48e9-88a3-b775c1b7806c admin admin] RBAC: Authorizing `identity:create_grant(project_id=89676dbc35634ec7994040c93f693f7c, user_id=fd88c8e9166940d8a09657997e6214d6, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:13.207940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 799/1609] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:13 2022] PUT /identity/v3/projects/4ea632c21d7549c89749672a6f65996d/users/c3b964fb4e024354ba066b38537d2da9/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:13.208593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936ceb8-eb55-43f1-af8a-d2cebe678742 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:13.217990 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400ccb4-23a4-48e9-88a3-b775c1b7806c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:13.234446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 811/1610] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:13 2022] PUT /identity/v3/projects/89676dbc35634ec7994040c93f693f7c/users/fd88c8e9166940d8a09657997e6214d6/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:13.239349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed434305-2a1f-48d7-9b4a-d77ebf030a9e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.239632 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed434305-2a1f-48d7-9b4a-d77ebf030a9e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.239922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed434305-2a1f-48d7-9b4a-d77ebf030a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.259763 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ed434305-2a1f-48d7-9b4a-d77ebf030a9e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:13.261009 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ed434305-2a1f-48d7-9b4a-d77ebf030a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:13.284918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:13.290704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936ceb8-eb55-43f1-af8a-d2cebe678742 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:13.336026 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 812/1611] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:13.337735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0f2fb-dfee-4952-8847-b839222087bb None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.338030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0f2fb-dfee-4952-8847-b839222087bb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.338310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0f2fb-dfee-4952-8847-b839222087bb None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.341011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 813/1612] 10.222.0.52 () {54 vars in 961 bytes} [Thu May 19 13:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:54:13.341193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0f2fb-dfee-4952-8847-b839222087bb None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:13.359251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0f2fb-dfee-4952-8847-b839222087bb None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:13.364462 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea9cd1-a365-442c-86db-9c3f33d11478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:13.365177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea9cd1-a365-442c-86db-9c3f33d11478 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.365602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea9cd1-a365-442c-86db-9c3f33d11478 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.365886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea9cd1-a365-442c-86db-9c3f33d11478 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.366578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea9cd1-a365-442c-86db-9c3f33d11478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:13.368709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea9cd1-a365-442c-86db-9c3f33d11478 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:13.381497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67aba2-8767-43c0-a49e-8603c83596b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:13.382513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67aba2-8767-43c0-a49e-8603c83596b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.382813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67aba2-8767-43c0-a49e-8603c83596b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.383140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67aba2-8767-43c0-a49e-8603c83596b7 admin admin] PATH_INFO: `/v3/projects/4eed7a893f7b42edaadbf898ebe16cd6/users/00b21c3634cc41aca680152740644168/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.402399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 814/1613] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:13 2022] POST /identity/v3/users => generated 292 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:13.407147 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea9cd1-a365-442c-86db-9c3f33d11478 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:13.417713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea9cd1-a365-442c-86db-9c3f33d11478 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:13.421335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67aba2-8767-43c0-a49e-8603c83596b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eed7a893f7b42edaadbf898ebe16cd6, user_id=00b21c3634cc41aca680152740644168, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:13.423772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac2cfc-af03-4be5-952b-d417437b7057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:13.424520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac2cfc-af03-4be5-952b-d417437b7057 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.424811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac2cfc-af03-4be5-952b-d417437b7057 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.425092 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac2cfc-af03-4be5-952b-d417437b7057 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.425977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac2cfc-af03-4be5-952b-d417437b7057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:13.426280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67aba2-8767-43c0-a49e-8603c83596b7 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:13.427615 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac2cfc-af03-4be5-952b-d417437b7057 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:13.443281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 815/1614] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:13.443281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03a595-dd3e-4699-9578-201383645deb None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.443281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03a595-dd3e-4699-9578-201383645deb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.443281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03a595-dd3e-4699-9578-201383645deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.454012 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 800/1615] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:13 2022] PUT /identity/v3/projects/4eed7a893f7b42edaadbf898ebe16cd6/users/00b21c3634cc41aca680152740644168/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:13.464736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84a399-2501-49c3-8164-356c3c5c0c52 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.465139 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84a399-2501-49c3-8164-356c3c5c0c52 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.465534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84a399-2501-49c3-8164-356c3c5c0c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.473558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-bd03a595-dd3e-4699-9578-201383645deb None None] MFA Rules not processed for user `c3b964fb4e024354ba066b38537d2da9`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:13.475222 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-bd03a595-dd3e-4699-9578-201383645deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:13.494333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-dc84a399-2501-49c3-8164-356c3c5c0c52 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:13.495743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-dc84a399-2501-49c3-8164-356c3c5c0c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:13.561152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 816/1616] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:13.562062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03a595-dd3e-4699-9578-201383645deb None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:13.566682 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 801/1617] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:13.571600 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5434b-fa6b-4f10-9dc5-6c25ab047b06 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.571730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:13.571921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5434b-fa6b-4f10-9dc5-6c25ab047b06 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.572187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5434b-fa6b-4f10-9dc5-6c25ab047b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.608485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-7bc5434b-fa6b-4f10-9dc5-6c25ab047b06 None None] MFA Rules not processed for user `4f6093d0c1fc41d98c58bace81e3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:13.618432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-7bc5434b-fa6b-4f10-9dc5-6c25ab047b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:13.639815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03a595-dd3e-4699-9578-201383645deb None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:13.643858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e315c1-082d-4ad5-9cf6-eed913f50aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:13.644683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e315c1-082d-4ad5-9cf6-eed913f50aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.644967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e315c1-082d-4ad5-9cf6-eed913f50aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.645246 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e315c1-082d-4ad5-9cf6-eed913f50aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.656945 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e315c1-082d-4ad5-9cf6-eed913f50aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:13.658390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e315c1-082d-4ad5-9cf6-eed913f50aa7 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:13.670257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 817/1618] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:13.671281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e315c1-082d-4ad5-9cf6-eed913f50aa7 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:13.682885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e315c1-082d-4ad5-9cf6-eed913f50aa7 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:13.691578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1646daf-90d5-4b11-bb7f-2e30fbededc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:13.692246 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1646daf-90d5-4b11-bb7f-2e30fbededc2 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.692546 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1646daf-90d5-4b11-bb7f-2e30fbededc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.692824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1646daf-90d5-4b11-bb7f-2e30fbededc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.693561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1646daf-90d5-4b11-bb7f-2e30fbededc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:13.695267 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1646daf-90d5-4b11-bb7f-2e30fbededc2 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:13.711585 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 802/1619] 10.222.0.52 () {58 vars in 1058 bytes} [Thu May 19 13:54:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:13.712451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5434b-fa6b-4f10-9dc5-6c25ab047b06 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:13.724287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5434b-fa6b-4f10-9dc5-6c25ab047b06 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:13.727525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 818/1620] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:13.729275 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18b8ee-172b-438d-a5df-dadff1525980 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.729494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b5195-b67e-4e7e-91b7-e357c49123ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:13.729621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18b8ee-172b-438d-a5df-dadff1525980 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.729925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18b8ee-172b-438d-a5df-dadff1525980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.730546 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b5195-b67e-4e7e-91b7-e357c49123ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.730850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b5195-b67e-4e7e-91b7-e357c49123ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.731163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b5195-b67e-4e7e-91b7-e357c49123ac admin admin] PATH_INFO: `/v3/projects/8dda670ea595449998e3f41b236edfe1/users/81f6e3eba3f044c1947dae5708809bbe/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.749982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b5195-b67e-4e7e-91b7-e357c49123ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dda670ea595449998e3f41b236edfe1, user_id=81f6e3eba3f044c1947dae5708809bbe, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:13.752777 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-2b18b8ee-172b-438d-a5df-dadff1525980 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:13.753725 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b5195-b67e-4e7e-91b7-e357c49123ac admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:13.754302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2b18b8ee-172b-438d-a5df-dadff1525980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:13.756942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:13.770904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 819/1621] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:13.771933 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18b8ee-172b-438d-a5df-dadff1525980 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:13.773560 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2b18b8ee-172b-438d-a5df-dadff1525980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:13.784796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 803/1622] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:13 2022] PUT /identity/v3/projects/8dda670ea595449998e3f41b236edfe1/users/81f6e3eba3f044c1947dae5708809bbe/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:13.787210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324648e3-e861-4521-9334-f99b7630a9d8 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.787730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324648e3-e861-4521-9334-f99b7630a9d8 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.787988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18b8ee-172b-438d-a5df-dadff1525980 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:13.788541 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324648e3-e861-4521-9334-f99b7630a9d8 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.795566 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 804/1623] 10.222.0.52 () {54 vars in 961 bytes} [Thu May 19 13:54:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:54:13.796704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324648e3-e861-4521-9334-f99b7630a9d8 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:13.853385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f7563-654c-4921-af40-594c6edbc00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:13.854176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f7563-654c-4921-af40-594c6edbc00e service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.854523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f7563-654c-4921-af40-594c6edbc00e service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.854970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f7563-654c-4921-af40-594c6edbc00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.884910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324648e3-e861-4521-9334-f99b7630a9d8 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:13.892667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a84345-3098-447e-8de8-51c0a28ecf40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:13.893276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a84345-3098-447e-8de8-51c0a28ecf40 service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:13.893854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a84345-3098-447e-8de8-51c0a28ecf40 service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:13.894171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a84345-3098-447e-8de8-51c0a28ecf40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:13.953356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f7563-654c-4921-af40-594c6edbc00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:13.955120 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f7563-654c-4921-af40-594c6edbc00e service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:13.975308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 820/1624] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:13.977281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f7563-654c-4921-af40-594c6edbc00e service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:13.988059 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a84345-3098-447e-8de8-51c0a28ecf40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:13.989776 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a84345-3098-447e-8de8-51c0a28ecf40 service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:13.991348 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f7563-654c-4921-af40-594c6edbc00e service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:13.999591 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3171322-8b45-4804-8c44-4c5510a27f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:14.000814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3171322-8b45-4804-8c44-4c5510a27f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:14.001312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3171322-8b45-4804-8c44-4c5510a27f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:14.001807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3171322-8b45-4804-8c44-4c5510a27f02 admin admin] PATH_INFO: `/v3/projects/9fff41c3212144bc81f8e7ccb6ca30b1/users/3123d03a562048589fdc165c2d6a3910/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:14.005416 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 805/1625] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:14.008296 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a84345-3098-447e-8de8-51c0a28ecf40 service nova] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:14.032752 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a84345-3098-447e-8de8-51c0a28ecf40 service nova] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:14.039267 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8d5bb-69a8-41cc-aecf-307e947308e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:14.039942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d5bb-69a8-41cc-aecf-307e947308e3 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:14.040990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d5bb-69a8-41cc-aecf-307e947308e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:14.041286 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d5bb-69a8-41cc-aecf-307e947308e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:14.061392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3171322-8b45-4804-8c44-4c5510a27f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fff41c3212144bc81f8e7ccb6ca30b1, user_id=3123d03a562048589fdc165c2d6a3910, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:14.064852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3171322-8b45-4804-8c44-4c5510a27f02 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:14.072783 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8d5bb-69a8-41cc-aecf-307e947308e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:14.074419 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8d5bb-69a8-41cc-aecf-307e947308e3 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:14.102687 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 821/1626] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:13 2022] PUT /identity/v3/projects/9fff41c3212144bc81f8e7ccb6ca30b1/users/3123d03a562048589fdc165c2d6a3910/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:14.103955 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3171322-8b45-4804-8c44-4c5510a27f02 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:14.104548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 806/1627] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:14.110406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f159aa4-7c7e-4247-a14e-e68d72655c7d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:14.110942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f159aa4-7c7e-4247-a14e-e68d72655c7d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:14.111232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f159aa4-7c7e-4247-a14e-e68d72655c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:14.136456 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:14.159781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-1f159aa4-7c7e-4247-a14e-e68d72655c7d None None] MFA Rules not processed for user `4f6093d0c1fc41d98c58bace81e3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:14.162471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1f159aa4-7c7e-4247-a14e-e68d72655c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:14.243684 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3171322-8b45-4804-8c44-4c5510a27f02 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:14.262005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00604f-afdc-4538-a03c-c9611318ef0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:14.263709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00604f-afdc-4538-a03c-c9611318ef0d admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:14.264075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00604f-afdc-4538-a03c-c9611318ef0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:14.264431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00604f-afdc-4538-a03c-c9611318ef0d admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:14.265477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00604f-afdc-4538-a03c-c9611318ef0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:14.267680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00604f-afdc-4538-a03c-c9611318ef0d admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:14.296764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 822/1628] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:14.298589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00604f-afdc-4538-a03c-c9611318ef0d admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:14.320786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00604f-afdc-4538-a03c-c9611318ef0d admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:14.327190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 807/1629] 10.222.0.52 () {58 vars in 1058 bytes} [Thu May 19 13:54:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:14.327190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f159aa4-7c7e-4247-a14e-e68d72655c7d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:14.342171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9837c-93e0-4a69-bffa-8bc911e2214e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:14.343116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9837c-93e0-4a69-bffa-8bc911e2214e admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:14.343685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9837c-93e0-4a69-bffa-8bc911e2214e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:14.344426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9837c-93e0-4a69-bffa-8bc911e2214e admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:14.345446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9837c-93e0-4a69-bffa-8bc911e2214e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:14.347664 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9837c-93e0-4a69-bffa-8bc911e2214e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:14.361297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f159aa4-7c7e-4247-a14e-e68d72655c7d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:14.369854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dc610-c523-4963-8c5d-0356779032a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:14.370765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dc610-c523-4963-8c5d-0356779032a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:14.371073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dc610-c523-4963-8c5d-0356779032a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:14.373790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dc610-c523-4963-8c5d-0356779032a2 admin admin] PATH_INFO: `/v3/projects/8dda670ea595449998e3f41b236edfe1/users/81f6e3eba3f044c1947dae5708809bbe/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:14.380758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654dc610-c523-4963-8c5d-0356779032a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dda670ea595449998e3f41b236edfe1, user_id=81f6e3eba3f044c1947dae5708809bbe, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:14.383596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654dc610-c523-4963-8c5d-0356779032a2 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:14.384967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 823/1630] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:14.386028 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9837c-93e0-4a69-bffa-8bc911e2214e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:14.404798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:14.409591 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9837c-93e0-4a69-bffa-8bc911e2214e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:14.424754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 808/1631] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:14 2022] PUT /identity/v3/projects/8dda670ea595449998e3f41b236edfe1/users/81f6e3eba3f044c1947dae5708809bbe/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:14.425570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dc610-c523-4963-8c5d-0356779032a2 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:14.528846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541fec6-afc5-47fc-94d9-ccdd091b644e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:14.529860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541fec6-afc5-47fc-94d9-ccdd091b644e service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:14.529860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541fec6-afc5-47fc-94d9-ccdd091b644e service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:14.530089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541fec6-afc5-47fc-94d9-ccdd091b644e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:14.539191 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dc610-c523-4963-8c5d-0356779032a2 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:14.546291 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc759427-8c61-4a72-82cc-5e34891aa53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:14.547371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc759427-8c61-4a72-82cc-5e34891aa53f admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:14.547704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc759427-8c61-4a72-82cc-5e34891aa53f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:14.548020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc759427-8c61-4a72-82cc-5e34891aa53f admin admin] PATH_INFO: `/v3/projects/9fff41c3212144bc81f8e7ccb6ca30b1/users/3123d03a562048589fdc165c2d6a3910/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:14.551421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc759427-8c61-4a72-82cc-5e34891aa53f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fff41c3212144bc81f8e7ccb6ca30b1, user_id=3123d03a562048589fdc165c2d6a3910, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:14.554520 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc759427-8c61-4a72-82cc-5e34891aa53f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:14.577175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 809/1632] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:14 2022] PUT /identity/v3/projects/9fff41c3212144bc81f8e7ccb6ca30b1/users/3123d03a562048589fdc165c2d6a3910/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:14.577175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d648f-c862-4b23-ac88-cd8e624198a8 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:14.577175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d648f-c862-4b23-ac88-cd8e624198a8 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:14.577175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d648f-c862-4b23-ac88-cd8e624198a8 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:14.577175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 810/1633] 10.222.0.52 () {54 vars in 912 bytes} [Thu May 19 13:54:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:54:14.577175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b65af-6f4a-4eb0-aba8-8b6f0e63ccd0 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:14.577809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b65af-6f4a-4eb0-aba8-8b6f0e63ccd0 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:14.577809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b65af-6f4a-4eb0-aba8-8b6f0e63ccd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:14.634524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-696b65af-6f4a-4eb0-aba8-8b6f0e63ccd0 None None] MFA Rules not processed for user `81f6e3eba3f044c1947dae5708809bbe`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:14.636757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-696b65af-6f4a-4eb0-aba8-8b6f0e63ccd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:14.638390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541fec6-afc5-47fc-94d9-ccdd091b644e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:14.640183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541fec6-afc5-47fc-94d9-ccdd091b644e service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:14.669746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 824/1634] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:14.686570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14c3a0-0bfb-4086-9b5f-3d1a84de1f65 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:14.694963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14c3a0-0bfb-4086-9b5f-3d1a84de1f65 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:14.695270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14c3a0-0bfb-4086-9b5f-3d1a84de1f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:14.754577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-fe14c3a0-0bfb-4086-9b5f-3d1a84de1f65 None None] MFA Rules not processed for user `4f6093d0c1fc41d98c58bace81e3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:14.755870 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-fe14c3a0-0bfb-4086-9b5f-3d1a84de1f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:14.852721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 811/1635] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:14.854315 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589bec7-5014-4f43-9b86-5db32584f861 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:14.854505 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:14.854634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589bec7-5014-4f43-9b86-5db32584f861 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:14.854892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589bec7-5014-4f43-9b86-5db32584f861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:14.917373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a589bec7-5014-4f43-9b86-5db32584f861 None None] MFA Rules not processed for user `3123d03a562048589fdc165c2d6a3910`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:14.919184 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a589bec7-5014-4f43-9b86-5db32584f861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:14.922292 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 825/1636] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 13:54:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:14.923342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14c3a0-0bfb-4086-9b5f-3d1a84de1f65 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:15.036148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14c3a0-0bfb-4086-9b5f-3d1a84de1f65 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:15.043119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419b5d2-c6f4-4ecf-89bb-0e9e3a2b781a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:15.044613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419b5d2-c6f4-4ecf-89bb-0e9e3a2b781a service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:15.046101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419b5d2-c6f4-4ecf-89bb-0e9e3a2b781a service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:15.046786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419b5d2-c6f4-4ecf-89bb-0e9e3a2b781a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:15.078106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6419b5d2-c6f4-4ecf-89bb-0e9e3a2b781a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:15.079946 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6419b5d2-c6f4-4ecf-89bb-0e9e3a2b781a service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:15.092277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 826/1637] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:15.093414 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419b5d2-c6f4-4ecf-89bb-0e9e3a2b781a service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:15.103469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419b5d2-c6f4-4ecf-89bb-0e9e3a2b781a service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:15.115201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817900d-ef20-475e-9237-1f1c8785aab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:15.116475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817900d-ef20-475e-9237-1f1c8785aab1 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:15.117278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817900d-ef20-475e-9237-1f1c8785aab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:15.117715 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817900d-ef20-475e-9237-1f1c8785aab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:15.134264 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:15.141249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817900d-ef20-475e-9237-1f1c8785aab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:15.143950 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 812/1638] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:15.155140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817900d-ef20-475e-9237-1f1c8785aab1 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:15.183454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 827/1639] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:15.215970 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589bec7-5014-4f43-9b86-5db32584f861 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:15.229726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589bec7-5014-4f43-9b86-5db32584f861 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:15.235379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a334661-359d-487b-b6da-82286e154e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:15.236902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a334661-359d-487b-b6da-82286e154e30 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:15.237393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a334661-359d-487b-b6da-82286e154e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:15.237911 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a334661-359d-487b-b6da-82286e154e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:15.252321 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a334661-359d-487b-b6da-82286e154e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:15.256209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a334661-359d-487b-b6da-82286e154e30 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:15.273619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 813/1640] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:15 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:15.308315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967783d-54e0-45da-917a-c7f3a0f1c737 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:15.309716 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967783d-54e0-45da-917a-c7f3a0f1c737 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:15.310938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967783d-54e0-45da-917a-c7f3a0f1c737 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:15.313990 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 828/1641] 10.222.0.52 () {54 vars in 912 bytes} [Thu May 19 13:54:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:54:15.333486 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763480cc-5bb4-42c4-b522-e37331f477c9 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:15.334064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763480cc-5bb4-42c4-b522-e37331f477c9 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:15.334419 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763480cc-5bb4-42c4-b522-e37331f477c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:15.373241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-763480cc-5bb4-42c4-b522-e37331f477c9 None None] MFA Rules not processed for user `4f6093d0c1fc41d98c58bace81e3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:15.375918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-763480cc-5bb4-42c4-b522-e37331f477c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:15.410312 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 814/1642] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 13:54:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:15.479510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967783d-54e0-45da-917a-c7f3a0f1c737 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:15.489538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763480cc-5bb4-42c4-b522-e37331f477c9 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:15.503286 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763480cc-5bb4-42c4-b522-e37331f477c9 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:15.506790 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967783d-54e0-45da-917a-c7f3a0f1c737 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:15.516455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ecbe0-2426-4ff9-a1ca-42979721c045 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:15.517129 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ecbe0-2426-4ff9-a1ca-42979721c045 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:15.517415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ecbe0-2426-4ff9-a1ca-42979721c045 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:15.517690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ecbe0-2426-4ff9-a1ca-42979721c045 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:15.519854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6c327-ee99-4ff9-b327-d18eeea0541b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:15.520461 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6c327-ee99-4ff9-b327-d18eeea0541b service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:15.520740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6c327-ee99-4ff9-b327-d18eeea0541b service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:15.521019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6c327-ee99-4ff9-b327-d18eeea0541b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:15.545560 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6c327-ee99-4ff9-b327-d18eeea0541b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:15.547380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6c327-ee99-4ff9-b327-d18eeea0541b service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:15.578958 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 829/1643] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:15.644412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ecbe0-2426-4ff9-a1ca-42979721c045 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:15.646184 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ecbe0-2426-4ff9-a1ca-42979721c045 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:15.664967 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 815/1644] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:54:16.176527 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6c327-ee99-4ff9-b327-d18eeea0541b service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:16.185590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6c327-ee99-4ff9-b327-d18eeea0541b service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:16.202689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadc3d0-9f9c-4cfe-95bf-2e284bf2217d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:16.203380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadc3d0-9f9c-4cfe-95bf-2e284bf2217d service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:16.203664 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadc3d0-9f9c-4cfe-95bf-2e284bf2217d service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:16.203940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadc3d0-9f9c-4cfe-95bf-2e284bf2217d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:16.273907 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dadc3d0-9f9c-4cfe-95bf-2e284bf2217d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:16.275486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dadc3d0-9f9c-4cfe-95bf-2e284bf2217d service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:16.303835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 830/1645] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:54:16.625974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ecbe0-2426-4ff9-a1ca-42979721c045 service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:16.748866 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ecbe0-2426-4ff9-a1ca-42979721c045 service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:16.749427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195da4e1-9cea-4869-9119-a246f60cda09 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:16.749722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195da4e1-9cea-4869-9119-a246f60cda09 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:16.750002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195da4e1-9cea-4869-9119-a246f60cda09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:16.764638 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b51583-885b-4e5e-86ed-0db887d48e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:16.765640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b51583-885b-4e5e-86ed-0db887d48e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:16.765929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b51583-885b-4e5e-86ed-0db887d48e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:16.781608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b51583-885b-4e5e-86ed-0db887d48e7a admin admin] PATH_INFO: `/v3/users/264a78e35a654cfea772c2a1d4f8a0d9` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:16.785146 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b51583-885b-4e5e-86ed-0db887d48e7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=264a78e35a654cfea772c2a1d4f8a0d9)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:16.786207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: WARNING py.warnings [None req-a5b51583-885b-4e5e-86ed-0db887d48e7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:54:16.786207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: warnings.warn(msg) May 19 13:54:16.786207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: May 19 13:54:16.787493 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b51583-885b-4e5e-86ed-0db887d48e7a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:16.838586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-195da4e1-9cea-4869-9119-a246f60cda09 None None] MFA Rules not processed for user `fd88c8e9166940d8a09657997e6214d6`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:16.842415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-195da4e1-9cea-4869-9119-a246f60cda09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:16.913572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:16.947723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a5b51583-885b-4e5e-86ed-0db887d48e7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '264a78e35a654cfea772c2a1d4f8a0d9'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:16.970827 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a5b51583-885b-4e5e-86ed-0db887d48e7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '264a78e35a654cfea772c2a1d4f8a0d9'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:16.972223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a5b51583-885b-4e5e-86ed-0db887d48e7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '264a78e35a654cfea772c2a1d4f8a0d9'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:16.998998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a5b51583-885b-4e5e-86ed-0db887d48e7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '264a78e35a654cfea772c2a1d4f8a0d9'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:17.023120 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 831/1646] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:54:17.029653 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651da9e-c2e4-45ed-81d3-40c2f749de06 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:17.030279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651da9e-c2e4-45ed-81d3-40c2f749de06 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:17.030279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651da9e-c2e4-45ed-81d3-40c2f749de06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:17.039756 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a5b51583-885b-4e5e-86ed-0db887d48e7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '264a78e35a654cfea772c2a1d4f8a0d9'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:17.042655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:17.061891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 816/1647] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:54:16 2022] DELETE /identity/v3/users/264a78e35a654cfea772c2a1d4f8a0d9 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:17.067944 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c80f-0fce-440d-a3d1-239213d5094c None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:17.068490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c80f-0fce-440d-a3d1-239213d5094c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:17.069062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c80f-0fce-440d-a3d1-239213d5094c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:17.069287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-6651da9e-c2e4-45ed-81d3-40c2f749de06 None None] MFA Rules not processed for user `fd88c8e9166940d8a09657997e6214d6`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:17.070523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-6651da9e-c2e4-45ed-81d3-40c2f749de06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:17.113247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3bb3c80f-0fce-440d-a3d1-239213d5094c None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:17.116790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3bb3c80f-0fce-440d-a3d1-239213d5094c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:17.205996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 832/1648] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:17.214426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651da9e-c2e4-45ed-81d3-40c2f749de06 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:17.221925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-6651da9e-c2e4-45ed-81d3-40c2f749de06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:17.275000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 817/1649] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:54:17.286162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3c80f-0fce-440d-a3d1-239213d5094c None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:17.287784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3bb3c80f-0fce-440d-a3d1-239213d5094c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:17.393690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3c80f-0fce-440d-a3d1-239213d5094c None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:17.405805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009fbb5-aa0c-43f7-b724-66f34c79d5ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:17.406811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009fbb5-aa0c-43f7-b724-66f34c79d5ed service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:17.407840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009fbb5-aa0c-43f7-b724-66f34c79d5ed service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:17.408131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009fbb5-aa0c-43f7-b724-66f34c79d5ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:17.410846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651da9e-c2e4-45ed-81d3-40c2f749de06 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:17.426205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fac80f-916a-41cf-8539-65a0652f3a1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:17.426487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac80f-916a-41cf-8539-65a0652f3a1b service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:17.426791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac80f-916a-41cf-8539-65a0652f3a1b service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:17.428481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2009fbb5-aa0c-43f7-b724-66f34c79d5ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:17.429958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2009fbb5-aa0c-43f7-b724-66f34c79d5ed service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:17.431053 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac80f-916a-41cf-8539-65a0652f3a1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:17.438611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f2fac80f-916a-41cf-8539-65a0652f3a1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:17.456475 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 818/1650] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:17.457440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fac80f-916a-41cf-8539-65a0652f3a1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:17.459199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fac80f-916a-41cf-8539-65a0652f3a1b service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:17.489416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 833/1651] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:54:18.306679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009a20a-5f79-44b1-9dfb-2d6658ec6690 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:18.307245 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009a20a-5f79-44b1-9dfb-2d6658ec6690 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:18.307245 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009a20a-5f79-44b1-9dfb-2d6658ec6690 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:18.309138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 819/1652] 10.222.0.52 () {54 vars in 912 bytes} [Thu May 19 13:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:54:18.326711 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5582426-21bf-4189-ae0e-15d8c6619849 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:18.327314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5582426-21bf-4189-ae0e-15d8c6619849 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:18.327389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5582426-21bf-4189-ae0e-15d8c6619849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:18.406703 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-a5582426-21bf-4189-ae0e-15d8c6619849 None None] MFA Rules not processed for user `f18f635bff0142409e90da66547927f1`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:18.408102 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a5582426-21bf-4189-ae0e-15d8c6619849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:18.430375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 834/1653] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 13:54:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:18.431859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009a20a-5f79-44b1-9dfb-2d6658ec6690 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:18.437064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a009a20a-5f79-44b1-9dfb-2d6658ec6690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:18.459690 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5582426-21bf-4189-ae0e-15d8c6619849 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:18.481700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009a20a-5f79-44b1-9dfb-2d6658ec6690 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:18.482143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5582426-21bf-4189-ae0e-15d8c6619849 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:18.486691 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad902c6-0343-4778-9d97-42a410c3c031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:18.486838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb78d78-ba2e-4e7e-b995-e8b0e207e386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:18.487425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb78d78-ba2e-4e7e-b995-e8b0e207e386 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:18.487608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad902c6-0343-4778-9d97-42a410c3c031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:18.487760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb78d78-ba2e-4e7e-b995-e8b0e207e386 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:18.487893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad902c6-0343-4778-9d97-42a410c3c031 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:18.488045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb78d78-ba2e-4e7e-b995-e8b0e207e386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:18.488171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad902c6-0343-4778-9d97-42a410c3c031 admin admin] PATH_INFO: `/v3/projects/d5a0c47cbf434c64a59b1ce21027ce70` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:18.489895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad902c6-0343-4778-9d97-42a410c3c031 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a0c47cbf434c64a59b1ce21027ce70)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:18.491856 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad902c6-0343-4778-9d97-42a410c3c031 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:18.498358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb78d78-ba2e-4e7e-b995-e8b0e207e386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:18.499844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb78d78-ba2e-4e7e-b995-e8b0e207e386 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:18.529256 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 835/1654] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:18.610418 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-aad902c6-0343-4778-9d97-42a410c3c031 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a0c47cbf434c64a59b1ce21027ce70', 'request_id': 'req-aad902c6-0343-4778-9d97-42a410c3c031'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:18.617670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-aad902c6-0343-4778-9d97-42a410c3c031 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a0c47cbf434c64a59b1ce21027ce70', 'request_id': 'req-aad902c6-0343-4778-9d97-42a410c3c031'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:18.639721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-aad902c6-0343-4778-9d97-42a410c3c031 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a0c47cbf434c64a59b1ce21027ce70', 'request_id': 'req-aad902c6-0343-4778-9d97-42a410c3c031'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:18.657014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-aad902c6-0343-4778-9d97-42a410c3c031 admin admin] The token cache is being invalidate because project d5a0c47cbf434c64a59b1ce21027ce70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:54:18.657314 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-aad902c6-0343-4778-9d97-42a410c3c031 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:18.663170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-aad902c6-0343-4778-9d97-42a410c3c031 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:18.669383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 820/1655] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:54:18 2022] DELETE /identity/v3/projects/d5a0c47cbf434c64a59b1ce21027ce70 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:18.671041 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8dcca-dd0c-44f7-b800-2a9c2f82d670 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:18.671538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8dcca-dd0c-44f7-b800-2a9c2f82d670 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:18.672016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8dcca-dd0c-44f7-b800-2a9c2f82d670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:18.700761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ddd8dcca-dd0c-44f7-b800-2a9c2f82d670 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:18.702319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ddd8dcca-dd0c-44f7-b800-2a9c2f82d670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:18.757158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:18.856196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03015adf-875a-48be-bc07-e6def883915a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:18.858158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03015adf-875a-48be-bc07-e6def883915a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:18.858268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03015adf-875a-48be-bc07-e6def883915a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:18.860866 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 836/1656] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:54:18.910921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-03015adf-875a-48be-bc07-e6def883915a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:18.912240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-03015adf-875a-48be-bc07-e6def883915a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:18.956884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 821/1657] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:18.962319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8dcca-dd0c-44f7-b800-2a9c2f82d670 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:18.980226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8dcca-dd0c-44f7-b800-2a9c2f82d670 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:19.001253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a939d-19e6-48c1-a836-85c88a4dd41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:19.001946 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a939d-19e6-48c1-a836-85c88a4dd41b admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:19.002711 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a939d-19e6-48c1-a836-85c88a4dd41b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:19.005224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a939d-19e6-48c1-a836-85c88a4dd41b admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:19.006058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a939d-19e6-48c1-a836-85c88a4dd41b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:19.006328 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a939d-19e6-48c1-a836-85c88a4dd41b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:19.008046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a939d-19e6-48c1-a836-85c88a4dd41b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:19.045572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 837/1658] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:54:19.047405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03015adf-875a-48be-bc07-e6def883915a None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:19.068421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03015adf-875a-48be-bc07-e6def883915a None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:19.087344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd400c8-d3bb-4d03-aefb-946886a13d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:19.088902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd400c8-d3bb-4d03-aefb-946886a13d1c admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:19.089234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd400c8-d3bb-4d03-aefb-946886a13d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:19.089518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd400c8-d3bb-4d03-aefb-946886a13d1c admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:19.090917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd400c8-d3bb-4d03-aefb-946886a13d1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:19.093538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd400c8-d3bb-4d03-aefb-946886a13d1c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:19.164443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:19.197562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 822/1659] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:19 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:19.198828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a939d-19e6-48c1-a836-85c88a4dd41b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:19.213188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a939d-19e6-48c1-a836-85c88a4dd41b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:19.447427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ecfd3-fed1-4cab-b450-2750c676c0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:19.452183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ecfd3-fed1-4cab-b450-2750c676c0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:19.452956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ecfd3-fed1-4cab-b450-2750c676c0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:19.453904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ecfd3-fed1-4cab-b450-2750c676c0d3 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:19.461910 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ecfd3-fed1-4cab-b450-2750c676c0d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:19.471295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ecfd3-fed1-4cab-b450-2750c676c0d3 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:19.539032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 838/1660] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:19 2022] POST /identity/v3/users => generated 291 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:19.543370 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd400c8-d3bb-4d03-aefb-946886a13d1c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:19.558654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd400c8-d3bb-4d03-aefb-946886a13d1c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:19.570044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1b7bb-17f9-4c86-b3b4-e55cc52ae954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:19.570746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1b7bb-17f9-4c86-b3b4-e55cc52ae954 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:19.571065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1b7bb-17f9-4c86-b3b4-e55cc52ae954 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:19.575153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1b7bb-17f9-4c86-b3b4-e55cc52ae954 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:19.578523 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1b7bb-17f9-4c86-b3b4-e55cc52ae954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:19.584202 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1b7bb-17f9-4c86-b3b4-e55cc52ae954 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:19.595833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04aca8-7116-4ce2-9d51-82eaf4ca46b1 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:19.597168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04aca8-7116-4ce2-9d51-82eaf4ca46b1 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:19.597820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04aca8-7116-4ce2-9d51-82eaf4ca46b1 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:19.600719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 839/1661] 10.222.0.52 () {54 vars in 960 bytes} [Thu May 19 13:54:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:54:19.609048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c1e49-9726-4830-9f34-7454aa7a346e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:19.609760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c1e49-9726-4830-9f34-7454aa7a346e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:19.610384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c1e49-9726-4830-9f34-7454aa7a346e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:19.616362 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 823/1662] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:19.620491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1b7bb-17f9-4c86-b3b4-e55cc52ae954 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:19.648894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-dd9c1e49-9726-4830-9f34-7454aa7a346e None None] MFA Rules not processed for user `b105f8163ce84a9ba725e6b04d348446`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:19.650520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-dd9c1e49-9726-4830-9f34-7454aa7a346e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:19.652010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1b7bb-17f9-4c86-b3b4-e55cc52ae954 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:19.676210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509f04d-6e35-4e40-a099-d0f560fe5dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:19.678426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509f04d-6e35-4e40-a099-d0f560fe5dfb admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:19.678723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509f04d-6e35-4e40-a099-d0f560fe5dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:19.680395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509f04d-6e35-4e40-a099-d0f560fe5dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:19.681087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509f04d-6e35-4e40-a099-d0f560fe5dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:19.689106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509f04d-6e35-4e40-a099-d0f560fe5dfb admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:19.726950 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 824/1663] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:19.731882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509f04d-6e35-4e40-a099-d0f560fe5dfb admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:19.769145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509f04d-6e35-4e40-a099-d0f560fe5dfb admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:19.833668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a646f-7184-4bb1-80b4-840364645264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:19.834595 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a646f-7184-4bb1-80b4-840364645264 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:19.834898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a646f-7184-4bb1-80b4-840364645264 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:19.835179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a646f-7184-4bb1-80b4-840364645264 admin admin] PATH_INFO: `/v3/projects/aade61f4c66041a7b281a5d9342d305d/users/9539332b89b74ff2ad9d40e1f51515d0/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:19.885000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 840/1664] 10.222.0.52 () {58 vars in 1057 bytes} [Thu May 19 13:54:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:19.892740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c1e49-9726-4830-9f34-7454aa7a346e None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:19.914289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c1e49-9726-4830-9f34-7454aa7a346e None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:19.927898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a646f-7184-4bb1-80b4-840364645264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aade61f4c66041a7b281a5d9342d305d, user_id=9539332b89b74ff2ad9d40e1f51515d0, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:19.928411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564d66d-5ce7-4612-9648-0fc076e4fa9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b105f8163ce84a9ba725e6b04d348446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:19.929095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564d66d-5ce7-4612-9648-0fc076e4fa9a service placement] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:19.929378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564d66d-5ce7-4612-9648-0fc076e4fa9a service placement] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:19.929656 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564d66d-5ce7-4612-9648-0fc076e4fa9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:19.930591 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a646f-7184-4bb1-80b4-840364645264 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:19.934652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5564d66d-5ce7-4612-9648-0fc076e4fa9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:19.972601 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564d66d-5ce7-4612-9648-0fc076e4fa9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:19.974504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 825/1665] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:19 2022] PUT /identity/v3/projects/aade61f4c66041a7b281a5d9342d305d/users/9539332b89b74ff2ad9d40e1f51515d0/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:19.974706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564d66d-5ce7-4612-9648-0fc076e4fa9a service placement] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:19.983359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154c9fd-dedc-41cf-8c53-29853ab019c3 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:19.983646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154c9fd-dedc-41cf-8c53-29853ab019c3 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:19.983938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154c9fd-dedc-41cf-8c53-29853ab019c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:20.012633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 841/1666] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:54:20.045974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-1154c9fd-dedc-41cf-8c53-29853ab019c3 None None] MFA Rules not processed for user `9539332b89b74ff2ad9d40e1f51515d0`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:20.047205 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1154c9fd-dedc-41cf-8c53-29853ab019c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:20.197635 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 826/1667] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:20.203109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564d66d-5ce7-4612-9648-0fc076e4fa9a service placement] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:20.225887 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564d66d-5ce7-4612-9648-0fc076e4fa9a service placement] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:20.285698 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:20.351593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1e963-6b20-4d2d-90b0-02159435f1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:20.352302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1e963-6b20-4d2d-90b0-02159435f1b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:20.352593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1e963-6b20-4d2d-90b0-02159435f1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:20.352884 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1e963-6b20-4d2d-90b0-02159435f1b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:20.353598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1e963-6b20-4d2d-90b0-02159435f1b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:20.360832 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1e963-6b20-4d2d-90b0-02159435f1b8 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:20.447717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 842/1668] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:20 2022] POST /identity/v3/projects => generated 379 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:54:20.452769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154c9fd-dedc-41cf-8c53-29853ab019c3 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:20.471758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154c9fd-dedc-41cf-8c53-29853ab019c3 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:20.620528 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd898e-9760-45c8-a24e-ec795eae629c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:20.622686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd898e-9760-45c8-a24e-ec795eae629c admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:20.623014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd898e-9760-45c8-a24e-ec795eae629c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:20.623498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd898e-9760-45c8-a24e-ec795eae629c admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:20.624214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd898e-9760-45c8-a24e-ec795eae629c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:20.626148 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd898e-9760-45c8-a24e-ec795eae629c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:20.691709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 827/1669] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:20 2022] POST /identity/v3/users => generated 296 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:20.692694 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1e963-6b20-4d2d-90b0-02159435f1b8 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:20.712139 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1e963-6b20-4d2d-90b0-02159435f1b8 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:20.732508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91e19f-e09b-4230-8773-e23766dfd868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:20.733501 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91e19f-e09b-4230-8773-e23766dfd868 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:20.733990 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91e19f-e09b-4230-8773-e23766dfd868 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:20.735339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91e19f-e09b-4230-8773-e23766dfd868 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:20.736311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91e19f-e09b-4230-8773-e23766dfd868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:20.739899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91e19f-e09b-4230-8773-e23766dfd868 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:20.790120 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 843/1670] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:54:20.794763 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd898e-9760-45c8-a24e-ec795eae629c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:20.821943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd898e-9760-45c8-a24e-ec795eae629c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:20.839429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4cf8e-a468-46df-b0de-f31ccaf82668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:20.841311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4cf8e-a468-46df-b0de-f31ccaf82668 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:20.842918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4cf8e-a468-46df-b0de-f31ccaf82668 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:20.843971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4cf8e-a468-46df-b0de-f31ccaf82668 admin admin] PATH_INFO: `/v3/projects/acf910be66ff4f3e9a695dc433f53587/users/b7e6df908b61432da58356a7a0812199/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:20.878367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4cf8e-a468-46df-b0de-f31ccaf82668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf910be66ff4f3e9a695dc433f53587, user_id=b7e6df908b61432da58356a7a0812199, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:20.881986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4cf8e-a468-46df-b0de-f31ccaf82668 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:20.919898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 828/1671] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:20 2022] PUT /identity/v3/projects/acf910be66ff4f3e9a695dc433f53587/users/b7e6df908b61432da58356a7a0812199/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:20.921756 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91e19f-e09b-4230-8773-e23766dfd868 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:20.936069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91e19f-e09b-4230-8773-e23766dfd868 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:21.038045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ae446-d13b-4012-b243-a50f1add99be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:21.039584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ae446-d13b-4012-b243-a50f1add99be admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:21.040198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ae446-d13b-4012-b243-a50f1add99be admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:21.040627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ae446-d13b-4012-b243-a50f1add99be admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:21.042249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ae446-d13b-4012-b243-a50f1add99be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:21.044610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ae446-d13b-4012-b243-a50f1add99be admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:21.069471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 844/1672] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:21.070374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4cf8e-a468-46df-b0de-f31ccaf82668 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:21.078943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:21.085217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4cf8e-a468-46df-b0de-f31ccaf82668 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:21.095366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae29bf-43b0-43c4-83b3-a232834b624d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:21.110003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae29bf-43b0-43c4-83b3-a232834b624d admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:21.110641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae29bf-43b0-43c4-83b3-a232834b624d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:21.111034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae29bf-43b0-43c4-83b3-a232834b624d admin admin] PATH_INFO: `/v3/projects/acf910be66ff4f3e9a695dc433f53587/users/b7e6df908b61432da58356a7a0812199/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:21.114715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae29bf-43b0-43c4-83b3-a232834b624d admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf910be66ff4f3e9a695dc433f53587, user_id=b7e6df908b61432da58356a7a0812199, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:21.117898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae29bf-43b0-43c4-83b3-a232834b624d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:21.140138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 829/1673] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:21 2022] PUT /identity/v3/projects/acf910be66ff4f3e9a695dc433f53587/users/b7e6df908b61432da58356a7a0812199/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:21.143018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95806c55-5002-4345-87c2-19d8794f85c8 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:21.144114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95806c55-5002-4345-87c2-19d8794f85c8 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:21.144114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95806c55-5002-4345-87c2-19d8794f85c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:21.183506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-95806c55-5002-4345-87c2-19d8794f85c8 None None] MFA Rules not processed for user `b7e6df908b61432da58356a7a0812199`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:21.186029 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-95806c55-5002-4345-87c2-19d8794f85c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:21.314527 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 845/1674] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:21.321801 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae29bf-43b0-43c4-83b3-a232834b624d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:21.323586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-14ae29bf-43b0-43c4-83b3-a232834b624d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:21.425016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae29bf-43b0-43c4-83b3-a232834b624d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:21.429158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a29da3-5ceb-47e8-bcdb-9a296858eafe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:21.430442 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29da3-5ceb-47e8-bcdb-9a296858eafe service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:21.430823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29da3-5ceb-47e8-bcdb-9a296858eafe service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:21.431227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29da3-5ceb-47e8-bcdb-9a296858eafe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:21.444064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a29da3-5ceb-47e8-bcdb-9a296858eafe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:21.446346 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a29da3-5ceb-47e8-bcdb-9a296858eafe service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:21.497537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 830/1675] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:22.037792 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95806c55-5002-4345-87c2-19d8794f85c8 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:22.049390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95806c55-5002-4345-87c2-19d8794f85c8 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:22.055225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb25b4-f3b7-4b14-b8da-85ef3e2440fa tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7e6df908b61432da58356a7a0812199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acf910be66ff4f3e9a695dc433f53587', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:22.056410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb25b4-f3b7-4b14-b8da-85ef3e2440fa tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:22.056897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb25b4-f3b7-4b14-b8da-85ef3e2440fa tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:22.057397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb25b4-f3b7-4b14-b8da-85ef3e2440fa tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:22.058342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb25b4-f3b7-4b14-b8da-85ef3e2440fa tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:22.058913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb25b4-f3b7-4b14-b8da-85ef3e2440fa tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:22.060973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb25b4-f3b7-4b14-b8da-85ef3e2440fa tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:22.080181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 846/1676] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:22.080848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a29da3-5ceb-47e8-bcdb-9a296858eafe service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:22.105310 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a29da3-5ceb-47e8-bcdb-9a296858eafe service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:22.111942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37836a64-9420-4d5f-8325-94de4283c108 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7e6df908b61432da58356a7a0812199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acf910be66ff4f3e9a695dc433f53587', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:22.112621 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37836a64-9420-4d5f-8325-94de4283c108 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:22.112907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37836a64-9420-4d5f-8325-94de4283c108 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:22.113185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37836a64-9420-4d5f-8325-94de4283c108 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:22.113889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37836a64-9420-4d5f-8325-94de4283c108 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:22.115841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37836a64-9420-4d5f-8325-94de4283c108 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:22.164858 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 831/1677] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:22 2022] POST /identity/v3/projects => generated 353 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:54:22.170087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb25b4-f3b7-4b14-b8da-85ef3e2440fa tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:22.186783 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb25b4-f3b7-4b14-b8da-85ef3e2440fa tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:22.197060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:22.255200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf5513-35ab-4146-9a06-687bc52e05a8 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7e6df908b61432da58356a7a0812199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acf910be66ff4f3e9a695dc433f53587', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:22.256144 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf5513-35ab-4146-9a06-687bc52e05a8 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:22.256144 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf5513-35ab-4146-9a06-687bc52e05a8 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:22.256431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf5513-35ab-4146-9a06-687bc52e05a8 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:22.257168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf5513-35ab-4146-9a06-687bc52e05a8 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:22.257483 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf5513-35ab-4146-9a06-687bc52e05a8 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:22.259335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf5513-35ab-4146-9a06-687bc52e05a8 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:22.273019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 847/1678] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:22.315216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37836a64-9420-4d5f-8325-94de4283c108 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:22.328096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37836a64-9420-4d5f-8325-94de4283c108 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:22.333961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7e6df908b61432da58356a7a0812199', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acf910be66ff4f3e9a695dc433f53587', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:22.334932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:22.335438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:22.335933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] PATH_INFO: `/v3/projects/8ba27a76e66b4b41ab068c10957004c7` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:22.336961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8ba27a76e66b4b41ab068c10957004c7)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:22.339036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:22.408063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba27a76e66b4b41ab068c10957004c7', 'request_id': 'req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:22.415816 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba27a76e66b4b41ab068c10957004c7', 'request_id': 'req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:22.424374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba27a76e66b4b41ab068c10957004c7', 'request_id': 'req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:22.438287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] The token cache is being invalidate because project 8ba27a76e66b4b41ab068c10957004c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:54:22.438742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:22.439556 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-71e8d99e-d1c9-4576-88af-1fdc1d82a8d1 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:22.441806 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 832/1679] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:54:22 2022] DELETE /identity/v3/projects/8ba27a76e66b4b41ab068c10957004c7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:22.945843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf5513-35ab-4146-9a06-687bc52e05a8 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:22.948130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-fadf5513-35ab-4146-9a06-687bc52e05a8 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:23.180739 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf5513-35ab-4146-9a06-687bc52e05a8 tempest-VolumeQuotaClassesTest-707756527 tempest-VolumeQuotaClassesTest-707756527-project-admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:23.349462 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:23.364643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:23.365033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:23.365520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] PATH_INFO: `/v3/users/9539332b89b74ff2ad9d40e1f51515d0` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:23.367177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9539332b89b74ff2ad9d40e1f51515d0)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:23.369477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:23.536140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9539332b89b74ff2ad9d40e1f51515d0'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:23.553585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9539332b89b74ff2ad9d40e1f51515d0'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:23.555217 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:23.568232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9539332b89b74ff2ad9d40e1f51515d0'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:23.569164 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9539332b89b74ff2ad9d40e1f51515d0'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:23.570589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9539332b89b74ff2ad9d40e1f51515d0'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:23.628440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 848/1680] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:54:22 2022] DELETE /identity/v3/users/9539332b89b74ff2ad9d40e1f51515d0 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:23.632691 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508946f-b36a-4414-9f10-f63db47c0ef8 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:23.636493 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508946f-b36a-4414-9f10-f63db47c0ef8 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:23.637168 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508946f-b36a-4414-9f10-f63db47c0ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:23.700373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-d508946f-b36a-4414-9f10-f63db47c0ef8 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:23.700373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d508946f-b36a-4414-9f10-f63db47c0ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:23.861311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:23.887062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 833/1681] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:54:23.923847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:23.926209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:24.061480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486068b-4a1d-49a7-83c3-350f22d7856f admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:24.066319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e24e18-227a-40c7-8a50-35f8eabc39c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:24.067190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e24e18-227a-40c7-8a50-35f8eabc39c0 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:24.067662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e24e18-227a-40c7-8a50-35f8eabc39c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:24.068046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e24e18-227a-40c7-8a50-35f8eabc39c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:24.086197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e24e18-227a-40c7-8a50-35f8eabc39c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:24.087758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e24e18-227a-40c7-8a50-35f8eabc39c0 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:24.114573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 849/1682] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:24.900013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ba98-e14c-4236-aa74-52f019e59963 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:24.900928 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ba98-e14c-4236-aa74-52f019e59963 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:24.901383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ba98-e14c-4236-aa74-52f019e59963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:25.145044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-74f7ba98-e14c-4236-aa74-52f019e59963 None None] MFA Rules not processed for user `698ccea06fb24beea18c679d53f907a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:25.146961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-74f7ba98-e14c-4236-aa74-52f019e59963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:25.285316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e24e18-227a-40c7-8a50-35f8eabc39c0 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:25.285316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-71e24e18-227a-40c7-8a50-35f8eabc39c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:25.286821 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 834/1683] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:25.288491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c6c20-2c71-4729-aea8-d7fb4157c682 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:25.288961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c6c20-2c71-4729-aea8-d7fb4157c682 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:25.289384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c6c20-2c71-4729-aea8-d7fb4157c682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:25.303777 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e24e18-227a-40c7-8a50-35f8eabc39c0 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:25.313275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-607c6c20-2c71-4729-aea8-d7fb4157c682 None None] MFA Rules not processed for user `698ccea06fb24beea18c679d53f907a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:25.314701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-607c6c20-2c71-4729-aea8-d7fb4157c682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:25.323092 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:25.324156 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:25.325410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:25.325637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] PATH_INFO: `/v3/projects/aade61f4c66041a7b281a5d9342d305d` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:25.326899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=aade61f4c66041a7b281a5d9342d305d)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:25.331163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 835/1684] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:25.332551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: WARNING py.warnings [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:54:25.332551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: warnings.warn(msg) May 19 13:54:25.332551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: May 19 13:54:25.333908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:25.337131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c6c20-2c71-4729-aea8-d7fb4157c682 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:25.340611 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-607c6c20-2c71-4729-aea8-d7fb4157c682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:25.460108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aade61f4c66041a7b281a5d9342d305d', 'request_id': 'req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:25.467795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c6c20-2c71-4729-aea8-d7fb4157c682 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:25.475070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aade61f4c66041a7b281a5d9342d305d', 'request_id': 'req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:25.502433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aade61f4c66041a7b281a5d9342d305d', 'request_id': 'req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:25.557836 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd16b7c-a577-4ff8-986a-ceb580b67014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:25.558656 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd16b7c-a577-4ff8-986a-ceb580b67014 service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:25.559205 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd16b7c-a577-4ff8-986a-ceb580b67014 service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:25.559591 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd16b7c-a577-4ff8-986a-ceb580b67014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:25.560087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] The token cache is being invalidate because project aade61f4c66041a7b281a5d9342d305d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:54:25.560368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:25.562887 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:25.569720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 850/1685] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:54:25 2022] DELETE /identity/v3/projects/aade61f4c66041a7b281a5d9342d305d => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:25.573729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:25.577487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:25.578356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:25.640537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd16b7c-a577-4ff8-986a-ceb580b67014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:25.643032 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd16b7c-a577-4ff8-986a-ceb580b67014 service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:25.655832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 836/1686] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:25.730704 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:25.739361 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9fbd7-74a6-47b0-b3c7-be10b2410bdf admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:25.760524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1ef95-d2bd-452c-a525-6976c0ff88b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:25.764579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1ef95-d2bd-452c-a525-6976c0ff88b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:25.764974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1ef95-d2bd-452c-a525-6976c0ff88b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:25.765358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1ef95-d2bd-452c-a525-6976c0ff88b6 admin admin] PATH_INFO: `/v3/users/b7e6df908b61432da58356a7a0812199` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:25.768018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1ef95-d2bd-452c-a525-6976c0ff88b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e6df908b61432da58356a7a0812199)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:25.773495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1ef95-d2bd-452c-a525-6976c0ff88b6 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:25.923446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-05f1ef95-d2bd-452c-a525-6976c0ff88b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e6df908b61432da58356a7a0812199'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:25.951382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-05f1ef95-d2bd-452c-a525-6976c0ff88b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e6df908b61432da58356a7a0812199'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:25.963249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-05f1ef95-d2bd-452c-a525-6976c0ff88b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e6df908b61432da58356a7a0812199'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:25.964310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-05f1ef95-d2bd-452c-a525-6976c0ff88b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e6df908b61432da58356a7a0812199'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:25.965599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-05f1ef95-d2bd-452c-a525-6976c0ff88b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e6df908b61432da58356a7a0812199'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:26.017444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 851/1687] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:54:25 2022] DELETE /identity/v3/users/b7e6df908b61432da58356a7a0812199 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:26.613493 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd16b7c-a577-4ff8-986a-ceb580b67014 service nova] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:26.631176 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd16b7c-a577-4ff8-986a-ceb580b67014 service nova] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:26.753575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:26.754514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:26.754886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:26.755399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179 admin admin] PATH_INFO: `/v3/projects/acf910be66ff4f3e9a695dc433f53587` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:26.756776 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acf910be66ff4f3e9a695dc433f53587)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:26.759896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:26.921432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acf910be66ff4f3e9a695dc433f53587', 'request_id': 'req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:26.938295 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acf910be66ff4f3e9a695dc433f53587', 'request_id': 'req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:26.957656 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acf910be66ff4f3e9a695dc433f53587', 'request_id': 'req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:26.974583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179 admin admin] The token cache is being invalidate because project acf910be66ff4f3e9a695dc433f53587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:54:26.975372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:26.976213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9667dbb2-90ba-4cc4-aad7-03c60b7ac179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:26.984244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 837/1688] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:54:26 2022] DELETE /identity/v3/projects/acf910be66ff4f3e9a695dc433f53587 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:26.986172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56c1a5-f511-4a8b-8c0b-2c2ba393aeb5 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:26.986630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56c1a5-f511-4a8b-8c0b-2c2ba393aeb5 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:26.986846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56c1a5-f511-4a8b-8c0b-2c2ba393aeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:27.028438 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-0d56c1a5-f511-4a8b-8c0b-2c2ba393aeb5 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:27.030048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-0d56c1a5-f511-4a8b-8c0b-2c2ba393aeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:27.186397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3762b12-9ba4-4809-b40f-7dc6ef62ecf0 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:27.186695 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3762b12-9ba4-4809-b40f-7dc6ef62ecf0 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:27.187010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3762b12-9ba4-4809-b40f-7dc6ef62ecf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:27.196014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 852/1689] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:27.228572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-c3762b12-9ba4-4809-b40f-7dc6ef62ecf0 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:27.234380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c3762b12-9ba4-4809-b40f-7dc6ef62ecf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:27.260344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06b859-b0cc-4370-bcdb-ff71ed8e70af None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:27.260647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06b859-b0cc-4370-bcdb-ff71ed8e70af None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:27.260950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06b859-b0cc-4370-bcdb-ff71ed8e70af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:27.301134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 838/1690] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:27.316252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3762b12-9ba4-4809-b40f-7dc6ef62ecf0 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:27.331092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3762b12-9ba4-4809-b40f-7dc6ef62ecf0 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:27.349669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-2d06b859-b0cc-4370-bcdb-ff71ed8e70af None None] MFA Rules not processed for user `00b21c3634cc41aca680152740644168`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:27.351596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2d06b859-b0cc-4370-bcdb-ff71ed8e70af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:27.357920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51bf07-2c91-4874-bd84-15344dd080af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:27.358809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51bf07-2c91-4874-bd84-15344dd080af admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:27.359138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51bf07-2c91-4874-bd84-15344dd080af admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:27.359419 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51bf07-2c91-4874-bd84-15344dd080af admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:27.360182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51bf07-2c91-4874-bd84-15344dd080af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:27.360613 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51bf07-2c91-4874-bd84-15344dd080af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:27.362149 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51bf07-2c91-4874-bd84-15344dd080af admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:27.413497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 839/1691] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:27.415933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51bf07-2c91-4874-bd84-15344dd080af admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:27.435485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51bf07-2c91-4874-bd84-15344dd080af admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:27.450729 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7d361-5724-41fc-97c4-160bc90f4a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:27.451624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7d361-5724-41fc-97c4-160bc90f4a73 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:27.451965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7d361-5724-41fc-97c4-160bc90f4a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:27.452843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7d361-5724-41fc-97c4-160bc90f4a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:27.453560 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7d361-5724-41fc-97c4-160bc90f4a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:27.455559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7d361-5724-41fc-97c4-160bc90f4a73 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:27.538095 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 840/1692] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:27 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:27.541466 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 853/1693] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:27.542421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7d361-5724-41fc-97c4-160bc90f4a73 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:27.559265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06b859-b0cc-4370-bcdb-ff71ed8e70af None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:27.560345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:27.564960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2d06b859-b0cc-4370-bcdb-ff71ed8e70af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:27.580709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:27.740494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7d361-5724-41fc-97c4-160bc90f4a73 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:27.758100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9969b-a212-4044-bf3f-c005c2cdc33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:27.758788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9969b-a212-4044-bf3f-c005c2cdc33b admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:27.759233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9969b-a212-4044-bf3f-c005c2cdc33b admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:27.762931 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9969b-a212-4044-bf3f-c005c2cdc33b admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:27.763452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9969b-a212-4044-bf3f-c005c2cdc33b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:27.767647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9969b-a212-4044-bf3f-c005c2cdc33b admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:27.820231 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 841/1694] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:27 2022] POST /identity/v3/users => generated 287 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:54:27.821285 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a4001-7c6f-4b51-859d-e44eb45cca6c None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:27.821542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06b859-b0cc-4370-bcdb-ff71ed8e70af None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:27.821646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a4001-7c6f-4b51-859d-e44eb45cca6c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:27.821846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a4001-7c6f-4b51-859d-e44eb45cca6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:27.873660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-389a4001-7c6f-4b51-859d-e44eb45cca6c None None] MFA Rules not processed for user `698ccea06fb24beea18c679d53f907a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:27.875039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-389a4001-7c6f-4b51-859d-e44eb45cca6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:27.987959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 842/1695] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:27.988972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a4001-7c6f-4b51-859d-e44eb45cca6c None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:27.994726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94434f78-f8fc-4107-b8dc-fe9ff59a8dd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:27.995482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94434f78-f8fc-4107-b8dc-fe9ff59a8dd0 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:27.995773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94434f78-f8fc-4107-b8dc-fe9ff59a8dd0 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:27.996065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94434f78-f8fc-4107-b8dc-fe9ff59a8dd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:28.024563 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a4001-7c6f-4b51-859d-e44eb45cca6c None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:28.041545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772b40f-da6b-4ade-8f07-862a0cbefe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:28.042233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b40f-da6b-4ade-8f07-862a0cbefe7c admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:28.042513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b40f-da6b-4ade-8f07-862a0cbefe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:28.042790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b40f-da6b-4ade-8f07-862a0cbefe7c admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:28.044173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772b40f-da6b-4ade-8f07-862a0cbefe7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:28.063175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772b40f-da6b-4ade-8f07-862a0cbefe7c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:28.082595 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 843/1696] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:28.084154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe70e1-a5af-425f-8720-616f455505ec None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:28.084452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe70e1-a5af-425f-8720-616f455505ec None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:28.084727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe70e1-a5af-425f-8720-616f455505ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:28.146713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94434f78-f8fc-4107-b8dc-fe9ff59a8dd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:28.149084 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94434f78-f8fc-4107-b8dc-fe9ff59a8dd0 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:28.190047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6cbe70e1-a5af-425f-8720-616f455505ec None None] MFA Rules not processed for user `0cc3e0105f2a472092423852b1f39979`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:28.191382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6cbe70e1-a5af-425f-8720-616f455505ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:28.191680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 854/1697] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:28.192958 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94434f78-f8fc-4107-b8dc-fe9ff59a8dd0 service nova] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:28.222966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94434f78-f8fc-4107-b8dc-fe9ff59a8dd0 service nova] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:28.237589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3621c9d-f904-405d-bc0f-5bfa59cb61ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:28.238278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621c9d-f904-405d-bc0f-5bfa59cb61ea admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:28.238576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621c9d-f904-405d-bc0f-5bfa59cb61ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:28.238833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621c9d-f904-405d-bc0f-5bfa59cb61ea admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:28.239591 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3621c9d-f904-405d-bc0f-5bfa59cb61ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:28.241189 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3621c9d-f904-405d-bc0f-5bfa59cb61ea admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:28.276583 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 855/1698] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:28.282115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3621c9d-f904-405d-bc0f-5bfa59cb61ea admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:28.303867 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3621c9d-f904-405d-bc0f-5bfa59cb61ea admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:28.330799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2815692-78a5-4858-a1cb-b17868af3fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:28.331962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2815692-78a5-4858-a1cb-b17868af3fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:28.331962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2815692-78a5-4858-a1cb-b17868af3fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:28.332267 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2815692-78a5-4858-a1cb-b17868af3fc8 admin admin] PATH_INFO: `/v3/projects/3eebac38d6a44d33a98e645610ba5b6f/users/55f9be5afb104c0abe01e1a7ed13d0c6/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:28.425792 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2815692-78a5-4858-a1cb-b17868af3fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eebac38d6a44d33a98e645610ba5b6f, user_id=55f9be5afb104c0abe01e1a7ed13d0c6, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:28.428560 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2815692-78a5-4858-a1cb-b17868af3fc8 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:28.497470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 844/1699] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:28.509593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 856/1700] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:28 2022] PUT /identity/v3/projects/3eebac38d6a44d33a98e645610ba5b6f/users/55f9be5afb104c0abe01e1a7ed13d0c6/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:28.521140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f5a3c-b494-48f6-88fa-c3784f633f77 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:28.521439 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f5a3c-b494-48f6-88fa-c3784f633f77 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:28.521721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f5a3c-b494-48f6-88fa-c3784f633f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:28.589727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-235f5a3c-b494-48f6-88fa-c3784f633f77 None None] MFA Rules not processed for user `55f9be5afb104c0abe01e1a7ed13d0c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:28.592191 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-235f5a3c-b494-48f6-88fa-c3784f633f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:28.598771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:28.755061 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2815692-78a5-4858-a1cb-b17868af3fc8 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:28.767813 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:28.771799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 845/1701] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:28.782097 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f5a3c-b494-48f6-88fa-c3784f633f77 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:28.786813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-235f5a3c-b494-48f6-88fa-c3784f633f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:28.998785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f5a3c-b494-48f6-88fa-c3784f633f77 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:29.026438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce831522-6a4d-4947-9107-ba1ce021fb6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:29.027186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce831522-6a4d-4947-9107-ba1ce021fb6e service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:29.028784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce831522-6a4d-4947-9107-ba1ce021fb6e service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:29.028784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce831522-6a4d-4947-9107-ba1ce021fb6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:29.031607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2815692-78a5-4858-a1cb-b17868af3fc8 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:29.050893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce831522-6a4d-4947-9107-ba1ce021fb6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:29.051926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce831522-6a4d-4947-9107-ba1ce021fb6e service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:29.053294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ecf21-44fa-4098-b905-61bd699a742b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:29.053963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ecf21-44fa-4098-b905-61bd699a742b service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:29.054244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ecf21-44fa-4098-b905-61bd699a742b service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:29.054882 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ecf21-44fa-4098-b905-61bd699a742b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:29.119476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 846/1702] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:29.150707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce831522-6a4d-4947-9107-ba1ce021fb6e service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:29.288772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ecf21-44fa-4098-b905-61bd699a742b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:29.291086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ecf21-44fa-4098-b905-61bd699a742b service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:29.297989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce831522-6a4d-4947-9107-ba1ce021fb6e service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:29.309461 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 857/1703] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:54:29.311336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5ce59-ea08-4e7c-9554-72723951f21a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:29.311628 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5ce59-ea08-4e7c-9554-72723951f21a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:29.312460 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5ce59-ea08-4e7c-9554-72723951f21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:29.322415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:29.323355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:29.323642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:29.323919 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] PATH_INFO: `/v3/users/55f9be5afb104c0abe01e1a7ed13d0c6` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:29.326534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=55f9be5afb104c0abe01e1a7ed13d0c6)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:29.328557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:29.424737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-54f5ce59-ea08-4e7c-9554-72723951f21a None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:29.450036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-54f5ce59-ea08-4e7c-9554-72723951f21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:29.563516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55f9be5afb104c0abe01e1a7ed13d0c6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:29.567025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:29.585079 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55f9be5afb104c0abe01e1a7ed13d0c6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:29.586777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55f9be5afb104c0abe01e1a7ed13d0c6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:29.620069 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55f9be5afb104c0abe01e1a7ed13d0c6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:29.685494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55f9be5afb104c0abe01e1a7ed13d0c6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:29.694037 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 858/1704] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 13:54:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:29.713936 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 847/1705] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:54:29 2022] DELETE /identity/v3/users/55f9be5afb104c0abe01e1a7ed13d0c6 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:29.718375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f6312-8e1b-41da-82df-b64376569a82 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:29.719623 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f6312-8e1b-41da-82df-b64376569a82 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:29.720559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f6312-8e1b-41da-82df-b64376569a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:29.778658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-6b0f6312-8e1b-41da-82df-b64376569a82 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:29.780243 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-6b0f6312-8e1b-41da-82df-b64376569a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:29.968724 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:30.027185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 859/1706] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:30.050555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:30.054325 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:30.171190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f6312-8e1b-41da-82df-b64376569a82 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:30.174098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-6b0f6312-8e1b-41da-82df-b64376569a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:30.551429 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f6312-8e1b-41da-82df-b64376569a82 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:30.553174 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e7df5-8440-4e82-955d-afcc0ecbc0cb admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:30.569944 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65569837-01e2-448a-a5a2-f86ae888e5df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:30.570669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65569837-01e2-448a-a5a2-f86ae888e5df service swift] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:30.570992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65569837-01e2-448a-a5a2-f86ae888e5df service swift] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:30.571354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65569837-01e2-448a-a5a2-f86ae888e5df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:30.573305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b47ed2-baf4-440e-95bc-114efbefdb75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:30.576621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b47ed2-baf4-440e-95bc-114efbefdb75 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:30.579749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-65569837-01e2-448a-a5a2-f86ae888e5df service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:30.579928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b47ed2-baf4-440e-95bc-114efbefdb75 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:30.580445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b47ed2-baf4-440e-95bc-114efbefdb75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:30.623329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b47ed2-baf4-440e-95bc-114efbefdb75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:30.631467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b47ed2-baf4-440e-95bc-114efbefdb75 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:30.686253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 860/1707] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:30.923003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65569837-01e2-448a-a5a2-f86ae888e5df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:30.936705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65569837-01e2-448a-a5a2-f86ae888e5df service swift] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:30.963342 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 848/1708] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 13:54:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 913 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:54:31.011578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b47ed2-baf4-440e-95bc-114efbefdb75 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:31.024578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-73b47ed2-baf4-440e-95bc-114efbefdb75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:31.121517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b47ed2-baf4-440e-95bc-114efbefdb75 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:31.156759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:31.157774 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:31.158644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:31.158644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] PATH_INFO: `/v3/users/c3b964fb4e024354ba066b38537d2da9` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:31.183094 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3b964fb4e024354ba066b38537d2da9)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:31.188496 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:31.494244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3b964fb4e024354ba066b38537d2da9'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:31.532217 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3b964fb4e024354ba066b38537d2da9'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:31.572693 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3b964fb4e024354ba066b38537d2da9'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:31.579609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3b964fb4e024354ba066b38537d2da9'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:31.582310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3b964fb4e024354ba066b38537d2da9'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:31.696747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 861/1709] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:54:31 2022] DELETE /identity/v3/users/c3b964fb4e024354ba066b38537d2da9 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:31.698634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638607d3-d78a-452e-934d-7654e42f99cf None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:31.701144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638607d3-d78a-452e-934d-7654e42f99cf None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:31.701655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638607d3-d78a-452e-934d-7654e42f99cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:31.780096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-638607d3-d78a-452e-934d-7654e42f99cf None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:31.792511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-638607d3-d78a-452e-934d-7654e42f99cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:32.103586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 849/1710] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:32.189886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:32.192555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:32.317008 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638607d3-d78a-452e-934d-7654e42f99cf None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:32.321050 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-638607d3-d78a-452e-934d-7654e42f99cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:32.356566 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638607d3-d78a-452e-934d-7654e42f99cf None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:32.378618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:32.379652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:32.379940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:32.380218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] PATH_INFO: `/v3/users/947e6f420c0d490699fb047c4e4a6dad` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:32.385712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] RBAC: Authorizing `identity:delete_user(user_id=947e6f420c0d490699fb047c4e4a6dad)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:32.393552 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd4c07-36e6-4f45-b298-6ce1932bd423 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:32.395061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:32.409116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:32.445890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12719cba-1e9e-467b-8cf1-be90b9f257cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:32.452387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12719cba-1e9e-467b-8cf1-be90b9f257cd service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:32.452789 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12719cba-1e9e-467b-8cf1-be90b9f257cd service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:32.453080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12719cba-1e9e-467b-8cf1-be90b9f257cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:32.525064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12719cba-1e9e-467b-8cf1-be90b9f257cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:32.526649 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12719cba-1e9e-467b-8cf1-be90b9f257cd service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:32.626676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 862/1711] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:32.690134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947e6f420c0d490699fb047c4e4a6dad'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:32.716315 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947e6f420c0d490699fb047c4e4a6dad'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:32.717964 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947e6f420c0d490699fb047c4e4a6dad'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:32.738780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947e6f420c0d490699fb047c4e4a6dad'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:32.819997 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947e6f420c0d490699fb047c4e4a6dad'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:32.857646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 850/1712] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:54:32 2022] DELETE /identity/v3/users/947e6f420c0d490699fb047c4e4a6dad => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:54:32.860894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353f77b-c3b1-4473-ac5b-ccdbbda551af None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:32.861377 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353f77b-c3b1-4473-ac5b-ccdbbda551af None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:32.861813 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353f77b-c3b1-4473-ac5b-ccdbbda551af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:32.903459 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:32.931441 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-9353f77b-c3b1-4473-ac5b-ccdbbda551af None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:32.933007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9353f77b-c3b1-4473-ac5b-ccdbbda551af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:33.091579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 863/1713] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:33.120272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:33.123087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:33.311707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fec7c4-61c6-4465-a7bf-421d25fecbce admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:33.329582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3aac6-fdc9-4c8e-9a08-d0de806b9d7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:33.333591 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3aac6-fdc9-4c8e-9a08-d0de806b9d7a service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:33.336252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3aac6-fdc9-4c8e-9a08-d0de806b9d7a service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:33.338417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3aac6-fdc9-4c8e-9a08-d0de806b9d7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:33.398644 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3aac6-fdc9-4c8e-9a08-d0de806b9d7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:33.406208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3aac6-fdc9-4c8e-9a08-d0de806b9d7a service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:33.457450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 851/1714] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:35.608646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353f77b-c3b1-4473-ac5b-ccdbbda551af None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:35.613782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9353f77b-c3b1-4473-ac5b-ccdbbda551af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:35.638487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353f77b-c3b1-4473-ac5b-ccdbbda551af None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:35.649938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:35.651804 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:35.655949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:35.657929 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] PATH_INFO: `/v3/projects/4ea632c21d7549c89749672a6f65996d` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:35.664290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ea632c21d7549c89749672a6f65996d)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:35.674910 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:35.795473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ea632c21d7549c89749672a6f65996d', 'request_id': 'req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:35.805904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ea632c21d7549c89749672a6f65996d', 'request_id': 'req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:35.828515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ea632c21d7549c89749672a6f65996d', 'request_id': 'req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:35.851526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] The token cache is being invalidate because project 4ea632c21d7549c89749672a6f65996d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:54:35.851821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:35.852603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:35.858911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 864/1715] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:54:35 2022] DELETE /identity/v3/projects/4ea632c21d7549c89749672a6f65996d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:35.860761 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856e434-4c93-4d1d-b6f3-2a0765314068 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:35.861250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856e434-4c93-4d1d-b6f3-2a0765314068 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:35.861339 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856e434-4c93-4d1d-b6f3-2a0765314068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:35.910301 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:35.913432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:35.923118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a856e434-4c93-4d1d-b6f3-2a0765314068 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:35.929363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a856e434-4c93-4d1d-b6f3-2a0765314068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:36.156583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 852/1716] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:36.165333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f6937-441f-4460-9088-1dc5a8dcd16d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:36.165843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f6937-441f-4460-9088-1dc5a8dcd16d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:36.166406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f6937-441f-4460-9088-1dc5a8dcd16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:36.191221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f72c1e-3bff-470f-bf56-2ea09bdf90cf admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:36.211527 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926550a-a65d-44c1-9da7-b5d632323ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:36.212793 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926550a-a65d-44c1-9da7-b5d632323ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:36.213730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926550a-a65d-44c1-9da7-b5d632323ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:36.214220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926550a-a65d-44c1-9da7-b5d632323ce8 admin admin] PATH_INFO: `/v3/projects/3eebac38d6a44d33a98e645610ba5b6f` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:36.216795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926550a-a65d-44c1-9da7-b5d632323ce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eebac38d6a44d33a98e645610ba5b6f)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:36.220538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926550a-a65d-44c1-9da7-b5d632323ce8 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:36.230291 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a49f6937-441f-4460-9088-1dc5a8dcd16d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:36.231622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a49f6937-441f-4460-9088-1dc5a8dcd16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:36.256561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 853/1717] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:54:36.260915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f6937-441f-4460-9088-1dc5a8dcd16d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:36.272550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f6937-441f-4460-9088-1dc5a8dcd16d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:36.285856 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c0fbb-d2c5-4999-b659-3e26a118f2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:36.286970 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c0fbb-d2c5-4999-b659-3e26a118f2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:36.287456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c0fbb-d2c5-4999-b659-3e26a118f2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:36.287927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c0fbb-d2c5-4999-b659-3e26a118f2e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:36.288840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c0fbb-d2c5-4999-b659-3e26a118f2e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:36.289335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c0fbb-d2c5-4999-b659-3e26a118f2e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:36.291543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c0fbb-d2c5-4999-b659-3e26a118f2e7 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:36.317081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 854/1718] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:36.328686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c0fbb-d2c5-4999-b659-3e26a118f2e7 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:36.350631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:36.359210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:36.368273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c0fbb-d2c5-4999-b659-3e26a118f2e7 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:36.414098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d926550a-a65d-44c1-9da7-b5d632323ce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eebac38d6a44d33a98e645610ba5b6f', 'request_id': 'req-d926550a-a65d-44c1-9da7-b5d632323ce8'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:36.444471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d926550a-a65d-44c1-9da7-b5d632323ce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eebac38d6a44d33a98e645610ba5b6f', 'request_id': 'req-d926550a-a65d-44c1-9da7-b5d632323ce8'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:36.460949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d926550a-a65d-44c1-9da7-b5d632323ce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eebac38d6a44d33a98e645610ba5b6f', 'request_id': 'req-d926550a-a65d-44c1-9da7-b5d632323ce8'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:36.497953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d926550a-a65d-44c1-9da7-b5d632323ce8 admin admin] The token cache is being invalidate because project 3eebac38d6a44d33a98e645610ba5b6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:54:36.498296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d926550a-a65d-44c1-9da7-b5d632323ce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:36.503605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d926550a-a65d-44c1-9da7-b5d632323ce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:36.508921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 865/1719] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:54:35 2022] DELETE /identity/v3/projects/3eebac38d6a44d33a98e645610ba5b6f => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:54:36.510550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a614391-e009-43bb-a8f8-01117a9c1bcf None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:36.510906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a614391-e009-43bb-a8f8-01117a9c1bcf None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:36.520104 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a614391-e009-43bb-a8f8-01117a9c1bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:36.570001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-2a614391-e009-43bb-a8f8-01117a9c1bcf None None] MFA Rules not processed for user `f18f635bff0142409e90da66547927f1`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:36.580882 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2a614391-e009-43bb-a8f8-01117a9c1bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:36.582902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9c4dc-7ab8-4e80-9d5b-5e0e81c8b7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:36.583557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9c4dc-7ab8-4e80-9d5b-5e0e81c8b7cb admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:36.583876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9c4dc-7ab8-4e80-9d5b-5e0e81c8b7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:36.584124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9c4dc-7ab8-4e80-9d5b-5e0e81c8b7cb admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:36.584896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9c4dc-7ab8-4e80-9d5b-5e0e81c8b7cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:36.602070 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9c4dc-7ab8-4e80-9d5b-5e0e81c8b7cb admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:36.737798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 855/1720] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:36 2022] POST /identity/v3/projects => generated 355 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:36.740343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a933-5c0c-4ae1-ba4d-d057553eb3ea None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:36.742582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a933-5c0c-4ae1-ba4d-d057553eb3ea None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:36.743456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a933-5c0c-4ae1-ba4d-d057553eb3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:36.747825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 866/1721] 10.222.0.65 () {58 vars in 1015 bytes} [Thu May 19 13:54:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:36.748761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a614391-e009-43bb-a8f8-01117a9c1bcf None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:36.752601 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2a614391-e009-43bb-a8f8-01117a9c1bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:36.795018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a614391-e009-43bb-a8f8-01117a9c1bcf None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:36.797629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3059a933-5c0c-4ae1-ba4d-d057553eb3ea None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:36.798904 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3059a933-5c0c-4ae1-ba4d-d057553eb3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:36.928424 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 856/1722] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:36.928424 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059a933-5c0c-4ae1-ba4d-d057553eb3ea None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:36.928424 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3059a933-5c0c-4ae1-ba4d-d057553eb3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:36.940805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059a933-5c0c-4ae1-ba4d-d057553eb3ea None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:36.946938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02ae36-ab15-43ff-b80b-53ff5fb6d62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:36.947641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02ae36-ab15-43ff-b80b-53ff5fb6d62e admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:36.948363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02ae36-ab15-43ff-b80b-53ff5fb6d62e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:36.948655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02ae36-ab15-43ff-b80b-53ff5fb6d62e admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:36.948782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:36.949413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02ae36-ab15-43ff-b80b-53ff5fb6d62e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:36.949664 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:36.949956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:36.950222 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] PATH_INFO: `/v3/projects/0b1f5c694f5f4ce1b8620ec6d85d8861` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:36.952223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02ae36-ab15-43ff-b80b-53ff5fb6d62e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:36.952768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1f5c694f5f4ce1b8620ec6d85d8861)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:36.955039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:36.984722 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 857/1723] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:36 2022] POST /identity/v3/users => generated 278 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:36.985799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02ae36-ab15-43ff-b80b-53ff5fb6d62e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:36.992904 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-8c02ae36-ab15-43ff-b80b-53ff5fb6d62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:37.071399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1f5c694f5f4ce1b8620ec6d85d8861', 'request_id': 'req-3e6fd4d8-e076-4146-a694-efd846ba41ba'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:37.083491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1f5c694f5f4ce1b8620ec6d85d8861', 'request_id': 'req-3e6fd4d8-e076-4146-a694-efd846ba41ba'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:37.108281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1f5c694f5f4ce1b8620ec6d85d8861', 'request_id': 'req-3e6fd4d8-e076-4146-a694-efd846ba41ba'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:37.128471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:37.146749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] The token cache is being invalidate because project 0b1f5c694f5f4ce1b8620ec6d85d8861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:54:37.147093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:37.149204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:37.151399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 867/1724] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:54:36 2022] DELETE /identity/v3/projects/0b1f5c694f5f4ce1b8620ec6d85d8861 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:37.153212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:37.156841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:37.318974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fd4d8-e076-4146-a694-efd846ba41ba admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:37.330728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1176538-ebca-443e-89c8-a5ec230902d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:37.331873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1176538-ebca-443e-89c8-a5ec230902d6 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:37.332162 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1176538-ebca-443e-89c8-a5ec230902d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:37.332507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1176538-ebca-443e-89c8-a5ec230902d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:37.341119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-b1176538-ebca-443e-89c8-a5ec230902d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:37.356802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02ae36-ab15-43ff-b80b-53ff5fb6d62e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:37.366368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1176538-ebca-443e-89c8-a5ec230902d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:37.368306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7cdc8-2644-430e-9ab6-f988251e4eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:37.368592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1176538-ebca-443e-89c8-a5ec230902d6 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:37.369839 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7cdc8-2644-430e-9ab6-f988251e4eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:37.370130 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7cdc8-2644-430e-9ab6-f988251e4eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:37.370418 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7cdc8-2644-430e-9ab6-f988251e4eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:37.396159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 868/1725] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:37.397736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff612a-923f-430a-bcd4-58778538a875 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:37.398134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff612a-923f-430a-bcd4-58778538a875 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:37.398421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff612a-923f-430a-bcd4-58778538a875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:37.406077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7cdc8-2644-430e-9ab6-f988251e4eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:37.417621 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7cdc8-2644-430e-9ab6-f988251e4eb8 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:37.456752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-73ff612a-923f-430a-bcd4-58778538a875 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:37.458989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-73ff612a-923f-430a-bcd4-58778538a875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:37.459298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 858/1726] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:37.460593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7cdc8-2644-430e-9ab6-f988251e4eb8 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:37.467468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-20a7cdc8-2644-430e-9ab6-f988251e4eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:37.550473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:37.663880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 869/1727] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:37.665280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff612a-923f-430a-bcd4-58778538a875 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:37.668827 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-73ff612a-923f-430a-bcd4-58778538a875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:37.702374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff612a-923f-430a-bcd4-58778538a875 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:37.723691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:37.725728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:37.726209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:37.726590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] PATH_INFO: `/v3/users/81f6e3eba3f044c1947dae5708809bbe` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:37.730656 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81f6e3eba3f044c1947dae5708809bbe)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:37.735210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:37.758693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7cdc8-2644-430e-9ab6-f988251e4eb8 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:37.786692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44b86d-21a5-437f-b28b-eaf5234b358e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:37.789687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44b86d-21a5-437f-b28b-eaf5234b358e admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:37.789808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44b86d-21a5-437f-b28b-eaf5234b358e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:37.790097 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44b86d-21a5-437f-b28b-eaf5234b358e admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:37.795728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44b86d-21a5-437f-b28b-eaf5234b358e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:37.802679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44b86d-21a5-437f-b28b-eaf5234b358e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:37.843435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81f6e3eba3f044c1947dae5708809bbe'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:37.850368 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 859/1728] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:37.850552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44b86d-21a5-437f-b28b-eaf5234b358e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:37.865026 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81f6e3eba3f044c1947dae5708809bbe'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:37.884464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44b86d-21a5-437f-b28b-eaf5234b358e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:37.891857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81f6e3eba3f044c1947dae5708809bbe'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:37.896065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81f6e3eba3f044c1947dae5708809bbe'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:37.896952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81f6e3eba3f044c1947dae5708809bbe'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:37.898169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b403cfb-9479-42f8-9669-f0fd32b86652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:37.900233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b403cfb-9479-42f8-9669-f0fd32b86652 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:37.901767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b403cfb-9479-42f8-9669-f0fd32b86652 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:37.902068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b403cfb-9479-42f8-9669-f0fd32b86652 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:37.904624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b403cfb-9479-42f8-9669-f0fd32b86652 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:37.904882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b403cfb-9479-42f8-9669-f0fd32b86652 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:37.911188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b403cfb-9479-42f8-9669-f0fd32b86652 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:37.953852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 860/1729] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:37.960243 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b403cfb-9479-42f8-9669-f0fd32b86652 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:37.960422 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 870/1730] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:54:37 2022] DELETE /identity/v3/users/81f6e3eba3f044c1947dae5708809bbe => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:37.961417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:37.963501 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:37.965573 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-4b403cfb-9479-42f8-9669-f0fd32b86652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:37.988360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:38.019142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b403cfb-9479-42f8-9669-f0fd32b86652 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:38.150344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48236c36-791a-4543-ad5c-95740b5be0f8 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:38.165486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557835d-1986-4b44-9f75-ce79262eaf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:38.166329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557835d-1986-4b44-9f75-ce79262eaf5e admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:38.167093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557835d-1986-4b44-9f75-ce79262eaf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:38.169798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557835d-1986-4b44-9f75-ce79262eaf5e admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:38.170802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557835d-1986-4b44-9f75-ce79262eaf5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:38.174349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557835d-1986-4b44-9f75-ce79262eaf5e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:38.212575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4aa37-c269-4440-8851-0025f6136b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:38.214159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4aa37-c269-4440-8851-0025f6136b4c admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:38.214748 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4aa37-c269-4440-8851-0025f6136b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:38.215352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4aa37-c269-4440-8851-0025f6136b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:38.228604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4aa37-c269-4440-8851-0025f6136b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:38.232158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4aa37-c269-4440-8851-0025f6136b4c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:38.254889 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 871/1731] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:37 2022] POST /identity/v3/projects => generated 369 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:38.259732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557835d-1986-4b44-9f75-ce79262eaf5e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:38.284150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 861/1732] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:38.285311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4aa37-c269-4440-8851-0025f6136b4c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:38.306715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4aa37-c269-4440-8851-0025f6136b4c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:38.467485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7bbe1-c0dc-4c54-a187-1d459affc13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:38.468671 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7bbe1-c0dc-4c54-a187-1d459affc13a admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:38.469100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7bbe1-c0dc-4c54-a187-1d459affc13a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:38.469542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7bbe1-c0dc-4c54-a187-1d459affc13a admin admin] PATH_INFO: `/v3/projects/5bf416054d854d7fb9ce1b270eaec194/users/f6104aa6b47f449a8c4174f1585d68ab/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:38.485394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557835d-1986-4b44-9f75-ce79262eaf5e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:38.496554 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21acb4-6ef2-492e-bc0c-aea0bf88fc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:38.497733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21acb4-6ef2-492e-bc0c-aea0bf88fc11 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:38.498202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21acb4-6ef2-492e-bc0c-aea0bf88fc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:38.498631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21acb4-6ef2-492e-bc0c-aea0bf88fc11 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:38.499575 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21acb4-6ef2-492e-bc0c-aea0bf88fc11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:38.505091 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21acb4-6ef2-492e-bc0c-aea0bf88fc11 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:38.513097 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7bbe1-c0dc-4c54-a187-1d459affc13a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf416054d854d7fb9ce1b270eaec194, user_id=f6104aa6b47f449a8c4174f1585d68ab, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:38.519687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7bbe1-c0dc-4c54-a187-1d459affc13a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:38.557257 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 862/1733] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:38 2022] PUT /identity/v3/projects/5bf416054d854d7fb9ce1b270eaec194/users/f6104aa6b47f449a8c4174f1585d68ab/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:38.558492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 872/1734] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:38 2022] POST /identity/v3/users => generated 285 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:38.566494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7bbe1-c0dc-4c54-a187-1d459affc13a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:38.576871 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb2f0a-5d76-425a-a937-27ce7b1c315b None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:38.577169 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb2f0a-5d76-425a-a937-27ce7b1c315b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:38.577452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb2f0a-5d76-425a-a937-27ce7b1c315b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:38.604506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:38.688187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-d8eb2f0a-5d76-425a-a937-27ce7b1c315b None None] MFA Rules not processed for user `f6104aa6b47f449a8c4174f1585d68ab`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:38.689460 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d8eb2f0a-5d76-425a-a937-27ce7b1c315b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:38.691986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7bbe1-c0dc-4c54-a187-1d459affc13a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:38.704462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f518f2-6caa-4cf3-bff6-23e45731c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:38.705238 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f518f2-6caa-4cf3-bff6-23e45731c84e admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:38.705597 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f518f2-6caa-4cf3-bff6-23e45731c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:38.706023 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f518f2-6caa-4cf3-bff6-23e45731c84e admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:38.707744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f518f2-6caa-4cf3-bff6-23e45731c84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:38.713064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f518f2-6caa-4cf3-bff6-23e45731c84e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:38.726823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 863/1735] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:38.730767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f518f2-6caa-4cf3-bff6-23e45731c84e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:38.751510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f518f2-6caa-4cf3-bff6-23e45731c84e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:38.758387 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5907d39-1859-4245-b979-b99b4b9d7ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:38.760498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5907d39-1859-4245-b979-b99b4b9d7ece admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:38.760789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5907d39-1859-4245-b979-b99b4b9d7ece admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:38.761068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5907d39-1859-4245-b979-b99b4b9d7ece admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:38.762966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5907d39-1859-4245-b979-b99b4b9d7ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:38.767843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5907d39-1859-4245-b979-b99b4b9d7ece admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:38.787320 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 864/1736] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:38.790619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5907d39-1859-4245-b979-b99b4b9d7ece admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:38.815431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5907d39-1859-4245-b979-b99b4b9d7ece admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:38.827933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc044c1-8c13-483c-899b-2d3a70dad334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:38.832501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc044c1-8c13-483c-899b-2d3a70dad334 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:38.835363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc044c1-8c13-483c-899b-2d3a70dad334 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:38.835759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc044c1-8c13-483c-899b-2d3a70dad334 admin admin] PATH_INFO: `/v3/projects/4fe46ffac52d412daf3663e2ab1273de/users/a9a3080f9aef4323952eed19ef9bb074/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:38.885346 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc044c1-8c13-483c-899b-2d3a70dad334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe46ffac52d412daf3663e2ab1273de, user_id=a9a3080f9aef4323952eed19ef9bb074, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:38.894904 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc044c1-8c13-483c-899b-2d3a70dad334 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:38.925267 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 873/1737] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:38 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:38.928971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb2f0a-5d76-425a-a937-27ce7b1c315b None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:38.939854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 865/1738] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:38 2022] PUT /identity/v3/projects/4fe46ffac52d412daf3663e2ab1273de/users/a9a3080f9aef4323952eed19ef9bb074/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:54:38.942341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b849ed-585d-42f1-963e-1dae58567ece None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:38.943014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b849ed-585d-42f1-963e-1dae58567ece None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:38.943914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b849ed-585d-42f1-963e-1dae58567ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:38.971479 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb2f0a-5d76-425a-a937-27ce7b1c315b None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:38.979854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-b2b849ed-585d-42f1-963e-1dae58567ece None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:38.982670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b2b849ed-585d-42f1-963e-1dae58567ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:39.052487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:39.096327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 866/1739] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:39.162506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369efe45-67ca-44d3-aab8-9ba122c87501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:39.163601 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369efe45-67ca-44d3-aab8-9ba122c87501 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:39.164557 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369efe45-67ca-44d3-aab8-9ba122c87501 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:39.164932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369efe45-67ca-44d3-aab8-9ba122c87501 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:39.165555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369efe45-67ca-44d3-aab8-9ba122c87501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:39.170768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369efe45-67ca-44d3-aab8-9ba122c87501 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:39.236051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b849ed-585d-42f1-963e-1dae58567ece None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:39.238122 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b2b849ed-585d-42f1-963e-1dae58567ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:39.241240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 874/1740] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:38 2022] POST /identity/v3/projects => generated 357 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:39.246675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369efe45-67ca-44d3-aab8-9ba122c87501 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:39.435774 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b849ed-585d-42f1-963e-1dae58567ece None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:39.474778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369efe45-67ca-44d3-aab8-9ba122c87501 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:39.499193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff38ff-de28-481f-aac3-09e1c3677aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:39.500340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff38ff-de28-481f-aac3-09e1c3677aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:39.500984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff38ff-de28-481f-aac3-09e1c3677aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:39.502324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff38ff-de28-481f-aac3-09e1c3677aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:39.503512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff38ff-de28-481f-aac3-09e1c3677aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:39.505856 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff38ff-de28-481f-aac3-09e1c3677aa5 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:39.529560 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:39.579418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 875/1741] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:39 2022] POST /identity/v3/users => generated 285 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:39.580950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff38ff-de28-481f-aac3-09e1c3677aa5 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:39.589184 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801457d-3b22-469f-96b8-4b5cc2e296bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:39.589951 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801457d-3b22-469f-96b8-4b5cc2e296bc service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:39.590247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801457d-3b22-469f-96b8-4b5cc2e296bc service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:39.590531 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801457d-3b22-469f-96b8-4b5cc2e296bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:39.608973 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff38ff-de28-481f-aac3-09e1c3677aa5 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:39.616883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef8c8a-f0e5-4b00-9356-8a45f20d0700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:39.617979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef8c8a-f0e5-4b00-9356-8a45f20d0700 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:39.618565 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef8c8a-f0e5-4b00-9356-8a45f20d0700 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:39.619105 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef8c8a-f0e5-4b00-9356-8a45f20d0700 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:39.620387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef8c8a-f0e5-4b00-9356-8a45f20d0700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:39.623674 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef8c8a-f0e5-4b00-9356-8a45f20d0700 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:39.631274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801457d-3b22-469f-96b8-4b5cc2e296bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:39.636316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801457d-3b22-469f-96b8-4b5cc2e296bc service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:39.664920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 876/1742] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:39.670566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef8c8a-f0e5-4b00-9356-8a45f20d0700 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:39.672476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 867/1743] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:39.686089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef8c8a-f0e5-4b00-9356-8a45f20d0700 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:39.692279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b703f3f-58f9-42b6-ad6c-57e4f32f0823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:39.695060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b703f3f-58f9-42b6-ad6c-57e4f32f0823 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:39.695931 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b703f3f-58f9-42b6-ad6c-57e4f32f0823 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:39.696493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b703f3f-58f9-42b6-ad6c-57e4f32f0823 admin admin] PATH_INFO: `/v3/projects/b27597fc470c4dba8ccb8d8a8d67b404/users/746227a53b0b4a3e9cf678aef0327f50/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:39.727024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b703f3f-58f9-42b6-ad6c-57e4f32f0823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27597fc470c4dba8ccb8d8a8d67b404, user_id=746227a53b0b4a3e9cf678aef0327f50, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:39.731376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b703f3f-58f9-42b6-ad6c-57e4f32f0823 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:39.762789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801457d-3b22-469f-96b8-4b5cc2e296bc service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:39.763362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 877/1744] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:39 2022] PUT /identity/v3/projects/b27597fc470c4dba8ccb8d8a8d67b404/users/746227a53b0b4a3e9cf678aef0327f50/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:39.778606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801457d-3b22-469f-96b8-4b5cc2e296bc service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:39.893947 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510adcf3-9802-4ab7-be63-16c5a8f37cfe None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:39.894620 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510adcf3-9802-4ab7-be63-16c5a8f37cfe None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:39.895039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510adcf3-9802-4ab7-be63-16c5a8f37cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:39.930897 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d81c2e-4588-4e8a-934c-10cd97e7ea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:39.931603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d81c2e-4588-4e8a-934c-10cd97e7ea3e admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:39.931992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d81c2e-4588-4e8a-934c-10cd97e7ea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:39.932299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d81c2e-4588-4e8a-934c-10cd97e7ea3e admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:39.933141 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d81c2e-4588-4e8a-934c-10cd97e7ea3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:39.935327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d81c2e-4588-4e8a-934c-10cd97e7ea3e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:39.938922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-510adcf3-9802-4ab7-be63-16c5a8f37cfe None None] MFA Rules not processed for user `0cc3e0105f2a472092423852b1f39979`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:39.944131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-510adcf3-9802-4ab7-be63-16c5a8f37cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:39.966381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 868/1745] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:54:39.972638 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d81c2e-4588-4e8a-934c-10cd97e7ea3e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:39.990147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d81c2e-4588-4e8a-934c-10cd97e7ea3e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:40.005167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2c60c-a4f6-4bb4-a253-3e94bf85059a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:40.006981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2c60c-a4f6-4bb4-a253-3e94bf85059a admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:40.007459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2c60c-a4f6-4bb4-a253-3e94bf85059a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:40.008118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2c60c-a4f6-4bb4-a253-3e94bf85059a admin admin] PATH_INFO: `/v3/projects/b27597fc470c4dba8ccb8d8a8d67b404/users/746227a53b0b4a3e9cf678aef0327f50/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:40.014805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2c60c-a4f6-4bb4-a253-3e94bf85059a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27597fc470c4dba8ccb8d8a8d67b404, user_id=746227a53b0b4a3e9cf678aef0327f50, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:40.025185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2c60c-a4f6-4bb4-a253-3e94bf85059a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:40.057143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 869/1746] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:39 2022] PUT /identity/v3/projects/b27597fc470c4dba8ccb8d8a8d67b404/users/746227a53b0b4a3e9cf678aef0327f50/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:54:40.067531 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60fc87-6c5b-4716-9b2e-2976847655eb None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:40.067764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60fc87-6c5b-4716-9b2e-2976847655eb None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:40.068033 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60fc87-6c5b-4716-9b2e-2976847655eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:40.127044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-cb60fc87-6c5b-4716-9b2e-2976847655eb None None] MFA Rules not processed for user `746227a53b0b4a3e9cf678aef0327f50`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:40.129348 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 878/1747] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:40.137230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-cb60fc87-6c5b-4716-9b2e-2976847655eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:40.350221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 870/1748] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:40 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:54:40.364612 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510adcf3-9802-4ab7-be63-16c5a8f37cfe None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:40.375850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-510adcf3-9802-4ab7-be63-16c5a8f37cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:40.384699 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:40.517857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510adcf3-9802-4ab7-be63-16c5a8f37cfe None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:40.659206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1cad9-c74d-4d64-98e4-ac8090925748 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:40.661313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1cad9-c74d-4d64-98e4-ac8090925748 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:40.662738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1cad9-c74d-4d64-98e4-ac8090925748 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:40.663187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1cad9-c74d-4d64-98e4-ac8090925748 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:40.693855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1cad9-c74d-4d64-98e4-ac8090925748 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:40.698088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1cad9-c74d-4d64-98e4-ac8090925748 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:40.737249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 879/1749] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:40 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:40.930497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60fc87-6c5b-4716-9b2e-2976847655eb None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:40.952258 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60fc87-6c5b-4716-9b2e-2976847655eb None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:40.962560 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393596b9-6b25-4dca-a089-878b43d16ef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:40.967341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393596b9-6b25-4dca-a089-878b43d16ef2 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:40.968310 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393596b9-6b25-4dca-a089-878b43d16ef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:40.968913 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393596b9-6b25-4dca-a089-878b43d16ef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:41.149002 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:41.202288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393596b9-6b25-4dca-a089-878b43d16ef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:41.212352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393596b9-6b25-4dca-a089-878b43d16ef2 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:41.257421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 871/1750] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:40 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:42.281215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703480e-a1e4-4c4b-a16c-2b858d11120e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:42.282493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703480e-a1e4-4c4b-a16c-2b858d11120e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:42.282793 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703480e-a1e4-4c4b-a16c-2b858d11120e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:42.353193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-a703480e-a1e4-4c4b-a16c-2b858d11120e None None] MFA Rules not processed for user `00b21c3634cc41aca680152740644168`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:42.359499 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a703480e-a1e4-4c4b-a16c-2b858d11120e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:42.636453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 880/1751] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:43.176569 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393596b9-6b25-4dca-a089-878b43d16ef2 service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:43.178655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-393596b9-6b25-4dca-a089-878b43d16ef2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:43.390390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393596b9-6b25-4dca-a089-878b43d16ef2 service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:43.397282 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526503d6-45c4-4a6a-b75d-bee97a135c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:43.400409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526503d6-45c4-4a6a-b75d-bee97a135c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:43.401220 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526503d6-45c4-4a6a-b75d-bee97a135c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:43.403349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526503d6-45c4-4a6a-b75d-bee97a135c9d admin admin] PATH_INFO: `/v3/projects/8dda670ea595449998e3f41b236edfe1` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:43.404856 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526503d6-45c4-4a6a-b75d-bee97a135c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dda670ea595449998e3f41b236edfe1)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:43.408256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526503d6-45c4-4a6a-b75d-bee97a135c9d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:43.459025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc72fd-7a1f-4a3c-a31d-80957d6631e5 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:43.462026 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc72fd-7a1f-4a3c-a31d-80957d6631e5 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:43.463561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc72fd-7a1f-4a3c-a31d-80957d6631e5 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:43.467456 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 881/1752] 10.222.0.52 () {54 vars in 921 bytes} [Thu May 19 13:54:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:54:43.516252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0235dc-68d4-406c-a8cd-6ef59f272479 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:43.518167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0235dc-68d4-406c-a8cd-6ef59f272479 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:43.518800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0235dc-68d4-406c-a8cd-6ef59f272479 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:43.520971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 882/1753] 10.222.0.52 () {54 vars in 912 bytes} [Thu May 19 13:54:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:54:43.529686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b1b21-d51a-4645-8ade-1c85f9353c0e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:43.530170 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b1b21-d51a-4645-8ade-1c85f9353c0e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:43.530613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b1b21-d51a-4645-8ade-1c85f9353c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:43.544479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-526503d6-45c4-4a6a-b75d-bee97a135c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dda670ea595449998e3f41b236edfe1', 'request_id': 'req-526503d6-45c4-4a6a-b75d-bee97a135c9d'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:43.566984 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-526503d6-45c4-4a6a-b75d-bee97a135c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dda670ea595449998e3f41b236edfe1', 'request_id': 'req-526503d6-45c4-4a6a-b75d-bee97a135c9d'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:43.572540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-c60b1b21-d51a-4645-8ade-1c85f9353c0e None None] MFA Rules not processed for user `f18f635bff0142409e90da66547927f1`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:43.584574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c60b1b21-d51a-4645-8ade-1c85f9353c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:43.588269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-526503d6-45c4-4a6a-b75d-bee97a135c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dda670ea595449998e3f41b236edfe1', 'request_id': 'req-526503d6-45c4-4a6a-b75d-bee97a135c9d'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:43.618266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-526503d6-45c4-4a6a-b75d-bee97a135c9d admin admin] The token cache is being invalidate because project 8dda670ea595449998e3f41b236edfe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:54:43.618587 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-526503d6-45c4-4a6a-b75d-bee97a135c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:43.625666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-526503d6-45c4-4a6a-b75d-bee97a135c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:54:43.635963 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 872/1754] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:54:43 2022] DELETE /identity/v3/projects/8dda670ea595449998e3f41b236edfe1 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:43.638274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0917f-72e3-4cd7-883b-ff2888613694 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:43.638595 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0917f-72e3-4cd7-883b-ff2888613694 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:43.638943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0917f-72e3-4cd7-883b-ff2888613694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:43.694379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-aee0917f-72e3-4cd7-883b-ff2888613694 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:43.695669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-aee0917f-72e3-4cd7-883b-ff2888613694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:43.702200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 883/1755] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 13:54:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 19 13:54:43.705784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130aa93-acc0-4a12-8e15-2b1b0637e5a2 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:43.706406 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130aa93-acc0-4a12-8e15-2b1b0637e5a2 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:43.706996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130aa93-acc0-4a12-8e15-2b1b0637e5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:43.749220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f130aa93-acc0-4a12-8e15-2b1b0637e5a2 None None] MFA Rules not processed for user `4f6093d0c1fc41d98c58bace81e3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:43.749220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f130aa93-acc0-4a12-8e15-2b1b0637e5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:43.783285 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:43.862551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 873/1756] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:54:43.867296 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0917f-72e3-4cd7-883b-ff2888613694 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:43.869739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-aee0917f-72e3-4cd7-883b-ff2888613694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:43.908378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0917f-72e3-4cd7-883b-ff2888613694 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:43.918740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 884/1757] 10.222.0.52 () {58 vars in 1018 bytes} [Thu May 19 13:54:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:43.920893 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2a4ba-bd02-4e4d-98d9-31f1aece74cb None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:43.921049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183e800-14aa-491f-8649-61298300027d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:43.921648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183e800-14aa-491f-8649-61298300027d service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:43.921774 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2a4ba-bd02-4e4d-98d9-31f1aece74cb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:43.921935 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183e800-14aa-491f-8649-61298300027d service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:43.922213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183e800-14aa-491f-8649-61298300027d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:43.922431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2a4ba-bd02-4e4d-98d9-31f1aece74cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:43.957878 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183e800-14aa-491f-8649-61298300027d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:43.959474 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183e800-14aa-491f-8649-61298300027d service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:43.964027 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-61c2a4ba-bd02-4e4d-98d9-31f1aece74cb None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:43.965888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-61c2a4ba-bd02-4e4d-98d9-31f1aece74cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:43.973211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 874/1758] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:43.998127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 885/1759] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:44.002887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183e800-14aa-491f-8649-61298300027d service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:44.016813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:44.026092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183e800-14aa-491f-8649-61298300027d service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:44.044419 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7eb65-b50d-440b-8d9d-bac17b03cc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:44.045729 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7eb65-b50d-440b-8d9d-bac17b03cc34 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:44.046117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7eb65-b50d-440b-8d9d-bac17b03cc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:44.046500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7eb65-b50d-440b-8d9d-bac17b03cc34 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:44.047726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7eb65-b50d-440b-8d9d-bac17b03cc34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:54:44.048180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7eb65-b50d-440b-8d9d-bac17b03cc34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:44.050101 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7eb65-b50d-440b-8d9d-bac17b03cc34 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:44.078516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 875/1760] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:54:44.082807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2a4ba-bd02-4e4d-98d9-31f1aece74cb None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:44.106636 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2a4ba-bd02-4e4d-98d9-31f1aece74cb None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:44.117150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a642c-464b-4df7-beae-04ed22e7a901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:44.122935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a642c-464b-4df7-beae-04ed22e7a901 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:44.131228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a642c-464b-4df7-beae-04ed22e7a901 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:44.132997 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a642c-464b-4df7-beae-04ed22e7a901 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:44.133765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a642c-464b-4df7-beae-04ed22e7a901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:44.139878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a642c-464b-4df7-beae-04ed22e7a901 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:44.229777 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 886/1761] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:44 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:44.234221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7eb65-b50d-440b-8d9d-bac17b03cc34 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:44.251435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7eb65-b50d-440b-8d9d-bac17b03cc34 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:44.255099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df0398-a374-4edf-be78-ee72063ecf5b None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:44.255434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df0398-a374-4edf-be78-ee72063ecf5b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:44.255764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df0398-a374-4edf-be78-ee72063ecf5b None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:44.263566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 887/1762] 10.222.0.52 () {54 vars in 921 bytes} [Thu May 19 13:54:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:54:44.433115 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba9545-27b3-4595-9ff4-c0b52859b08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:44.433877 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba9545-27b3-4595-9ff4-c0b52859b08c admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:44.434161 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba9545-27b3-4595-9ff4-c0b52859b08c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:44.434442 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba9545-27b3-4595-9ff4-c0b52859b08c admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:44.435243 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba9545-27b3-4595-9ff4-c0b52859b08c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:44.437100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba9545-27b3-4595-9ff4-c0b52859b08c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:44.439828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3e5fc-7ca4-42bf-8a84-82b6afa667c1 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:44.439961 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3e5fc-7ca4-42bf-8a84-82b6afa667c1 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:44.440285 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3e5fc-7ca4-42bf-8a84-82b6afa667c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:44.481574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e6d3e5fc-7ca4-42bf-8a84-82b6afa667c1 None None] MFA Rules not processed for user `4f6093d0c1fc41d98c58bace81e3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:44.489940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e6d3e5fc-7ca4-42bf-8a84-82b6afa667c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:44.496373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 876/1763] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:44 2022] POST /identity/v3/users => generated 289 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:44.499833 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba9545-27b3-4595-9ff4-c0b52859b08c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:44.539092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba9545-27b3-4595-9ff4-c0b52859b08c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:44.561585 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072b76e-f132-4475-b9a8-c294cb90b707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:44.562275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072b76e-f132-4475-b9a8-c294cb90b707 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:44.562565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072b76e-f132-4475-b9a8-c294cb90b707 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:44.562846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072b76e-f132-4475-b9a8-c294cb90b707 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:44.563643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072b76e-f132-4475-b9a8-c294cb90b707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:44.565254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072b76e-f132-4475-b9a8-c294cb90b707 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:44.592976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 877/1764] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:44.594019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072b76e-f132-4475-b9a8-c294cb90b707 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:44.615577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072b76e-f132-4475-b9a8-c294cb90b707 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:44.629535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a3bf9-09bd-46e9-a77f-a4642bf6c83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:44.630231 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a3bf9-09bd-46e9-a77f-a4642bf6c83f admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:44.630585 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a3bf9-09bd-46e9-a77f-a4642bf6c83f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:44.630796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a3bf9-09bd-46e9-a77f-a4642bf6c83f admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:44.631583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a3bf9-09bd-46e9-a77f-a4642bf6c83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:44.633212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a3bf9-09bd-46e9-a77f-a4642bf6c83f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:44.658551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 878/1765] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:44.668639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a3bf9-09bd-46e9-a77f-a4642bf6c83f admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:44.679231 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 888/1766] 10.222.0.52 () {58 vars in 1018 bytes} [Thu May 19 13:54:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:54:44.688042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a3bf9-09bd-46e9-a77f-a4642bf6c83f admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:44.702153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3b53-9c51-4769-bdef-9d1b07624159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:44.703134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3b53-9c51-4769-bdef-9d1b07624159 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:44.703414 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3b53-9c51-4769-bdef-9d1b07624159 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:44.703746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3b53-9c51-4769-bdef-9d1b07624159 admin admin] PATH_INFO: `/v3/projects/c5bc2b82879f47ce8132eb70cc0a440b/users/33bc2adb0ce64a27866e1369a8a31a23/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:44.753627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e3b53-9c51-4769-bdef-9d1b07624159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5bc2b82879f47ce8132eb70cc0a440b, user_id=33bc2adb0ce64a27866e1369a8a31a23, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:44.761124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e3b53-9c51-4769-bdef-9d1b07624159 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:44.803167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 879/1767] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:44 2022] PUT /identity/v3/projects/c5bc2b82879f47ce8132eb70cc0a440b/users/33bc2adb0ce64a27866e1369a8a31a23/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:54:44.818014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2759a40-1a0e-4fb8-b1c4-a6d0edad95c6 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:44.818458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2759a40-1a0e-4fb8-b1c4-a6d0edad95c6 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:44.818913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2759a40-1a0e-4fb8-b1c4-a6d0edad95c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:44.865608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-c2759a40-1a0e-4fb8-b1c4-a6d0edad95c6 None None] MFA Rules not processed for user `33bc2adb0ce64a27866e1369a8a31a23`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:44.877692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c2759a40-1a0e-4fb8-b1c4-a6d0edad95c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:45.010910 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 889/1768] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:45.016621 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3b53-9c51-4769-bdef-9d1b07624159 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:45.150447 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761d704-a868-4727-a8e9-924374179101 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:45.153245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761d704-a868-4727-a8e9-924374179101 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:45.155083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761d704-a868-4727-a8e9-924374179101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:45.165347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:45.175972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3b53-9c51-4769-bdef-9d1b07624159 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:45.187232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737262f6-92c5-4f67-8810-f453063f21ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:45.188306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737262f6-92c5-4f67-8810-f453063f21ab admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:45.188741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737262f6-92c5-4f67-8810-f453063f21ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:45.189169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737262f6-92c5-4f67-8810-f453063f21ab admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:45.196249 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737262f6-92c5-4f67-8810-f453063f21ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:45.198733 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737262f6-92c5-4f67-8810-f453063f21ab admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:45.206190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-5761d704-a868-4727-a8e9-924374179101 None None] MFA Rules not processed for user `a9a3080f9aef4323952eed19ef9bb074`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:45.207963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5761d704-a868-4727-a8e9-924374179101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:45.249293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 880/1769] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:54:45 2022] POST /identity/v3/projects => generated 377 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:45.252975 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737262f6-92c5-4f67-8810-f453063f21ab admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:45.282159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:45.297012 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 890/1770] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:45.310816 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761d704-a868-4727-a8e9-924374179101 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:45.312831 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5761d704-a868-4727-a8e9-924374179101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:45.349941 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737262f6-92c5-4f67-8810-f453063f21ab admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:45.359983 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82cb25-08e1-4c15-9787-f8a52f4680cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:45.362273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82cb25-08e1-4c15-9787-f8a52f4680cb admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:45.363679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82cb25-08e1-4c15-9787-f8a52f4680cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:45.364131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82cb25-08e1-4c15-9787-f8a52f4680cb admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:45.365445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82cb25-08e1-4c15-9787-f8a52f4680cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:45.367807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82cb25-08e1-4c15-9787-f8a52f4680cb admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:45.421950 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 881/1771] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:54:45 2022] POST /identity/v3/users => generated 295 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:54:45.425511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82cb25-08e1-4c15-9787-f8a52f4680cb admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:45.439058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761d704-a868-4727-a8e9-924374179101 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:45.440116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82cb25-08e1-4c15-9787-f8a52f4680cb admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:45.447755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038270d5-e2ba-453c-9fbd-e96b1f79f251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:45.448859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038270d5-e2ba-453c-9fbd-e96b1f79f251 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:45.450227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038270d5-e2ba-453c-9fbd-e96b1f79f251 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:45.450575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038270d5-e2ba-453c-9fbd-e96b1f79f251 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:45.451590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038270d5-e2ba-453c-9fbd-e96b1f79f251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:45.454591 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038270d5-e2ba-453c-9fbd-e96b1f79f251 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:45.473273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 882/1772] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:54:45.477116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038270d5-e2ba-453c-9fbd-e96b1f79f251 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:45.499132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038270d5-e2ba-453c-9fbd-e96b1f79f251 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:45.515978 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3497b-1d6a-42d6-a236-59321ee7717a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:45.517047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3497b-1d6a-42d6-a236-59321ee7717a admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:45.517626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3497b-1d6a-42d6-a236-59321ee7717a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:45.517626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3497b-1d6a-42d6-a236-59321ee7717a admin admin] PATH_INFO: `/v3/projects/1df27c12146e4fbeb2cf644d61010515/users/5a4586166f0d42deabc9b3e42d4f3356/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:45.540078 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf09fb-71f9-4e0c-83c2-7632a235740e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:45.540747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf09fb-71f9-4e0c-83c2-7632a235740e service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:45.541031 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf09fb-71f9-4e0c-83c2-7632a235740e service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:45.541303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf09fb-71f9-4e0c-83c2-7632a235740e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:45.553667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3497b-1d6a-42d6-a236-59321ee7717a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df27c12146e4fbeb2cf644d61010515, user_id=5a4586166f0d42deabc9b3e42d4f3356, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:45.556439 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3497b-1d6a-42d6-a236-59321ee7717a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:45.582656 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 883/1773] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:45 2022] PUT /identity/v3/projects/1df27c12146e4fbeb2cf644d61010515/users/5a4586166f0d42deabc9b3e42d4f3356/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:54:45.583967 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3497b-1d6a-42d6-a236-59321ee7717a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:45.630770 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cf09fb-71f9-4e0c-83c2-7632a235740e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:45.632244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cf09fb-71f9-4e0c-83c2-7632a235740e service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:45.642550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 891/1774] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:45 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:54:45.684560 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3497b-1d6a-42d6-a236-59321ee7717a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:45.692608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377481d3-4473-4bbd-8be4-1586cfc90049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:45.693430 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377481d3-4473-4bbd-8be4-1586cfc90049 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:45.693836 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377481d3-4473-4bbd-8be4-1586cfc90049 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:45.694399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377481d3-4473-4bbd-8be4-1586cfc90049 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:45.695300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377481d3-4473-4bbd-8be4-1586cfc90049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:45.697671 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377481d3-4473-4bbd-8be4-1586cfc90049 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:45.712411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 884/1775] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:54:45.717022 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf09fb-71f9-4e0c-83c2-7632a235740e service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:45.722984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:45.744939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf09fb-71f9-4e0c-83c2-7632a235740e service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:45.768091 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c058d-9220-44dc-b8ed-a7785c024d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:45.769710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c058d-9220-44dc-b8ed-a7785c024d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:45.770196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c058d-9220-44dc-b8ed-a7785c024d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:45.770761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c058d-9220-44dc-b8ed-a7785c024d0e admin admin] PATH_INFO: `/v3/projects/1df27c12146e4fbeb2cf644d61010515/users/5a4586166f0d42deabc9b3e42d4f3356/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:45.774265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c058d-9220-44dc-b8ed-a7785c024d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df27c12146e4fbeb2cf644d61010515, user_id=5a4586166f0d42deabc9b3e42d4f3356, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:45.779497 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c058d-9220-44dc-b8ed-a7785c024d0e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:45.795972 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 892/1776] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:54:45 2022] PUT /identity/v3/projects/1df27c12146e4fbeb2cf644d61010515/users/5a4586166f0d42deabc9b3e42d4f3356/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:54:45.806193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd702-3588-4635-86b2-a9071eeacd6a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:45.806489 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd702-3588-4635-86b2-a9071eeacd6a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:45.806771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd702-3588-4635-86b2-a9071eeacd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:45.865167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-e0abd702-3588-4635-86b2-a9071eeacd6a None None] MFA Rules not processed for user `5a4586166f0d42deabc9b3e42d4f3356`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:45.867314 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e0abd702-3588-4635-86b2-a9071eeacd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:45.913121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c058d-9220-44dc-b8ed-a7785c024d0e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:45.914809 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5f6c058d-9220-44dc-b8ed-a7785c024d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:46.002357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:54:46.034210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 885/1777] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:46.083440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c058d-9220-44dc-b8ed-a7785c024d0e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:46.093966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7c942-1d31-4514-ac7d-b6a312ce4bbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:46.095590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7c942-1d31-4514-ac7d-b6a312ce4bbd service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:46.096631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7c942-1d31-4514-ac7d-b6a312ce4bbd service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:46.097871 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7c942-1d31-4514-ac7d-b6a312ce4bbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:46.117959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abd702-3588-4635-86b2-a9071eeacd6a None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:46.119516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e0abd702-3588-4635-86b2-a9071eeacd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:46.123350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7c942-1d31-4514-ac7d-b6a312ce4bbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:46.125228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7c942-1d31-4514-ac7d-b6a312ce4bbd service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:46.138296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 893/1778] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:46.268336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7c942-1d31-4514-ac7d-b6a312ce4bbd service nova] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:46.271473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abd702-3588-4635-86b2-a9071eeacd6a None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:46.282135 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7c942-1d31-4514-ac7d-b6a312ce4bbd service nova] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:46.284366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d70217-bc72-44c4-968c-ed48520c895f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:46.285303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d70217-bc72-44c4-968c-ed48520c895f service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:46.286211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d70217-bc72-44c4-968c-ed48520c895f service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:46.287624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d70217-bc72-44c4-968c-ed48520c895f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:46.297424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb512f3-2745-48f2-ab4b-2e5e40c2e874 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:46.298269 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb512f3-2745-48f2-ab4b-2e5e40c2e874 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:46.298553 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb512f3-2745-48f2-ab4b-2e5e40c2e874 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:46.299017 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb512f3-2745-48f2-ab4b-2e5e40c2e874 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:46.334470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb512f3-2745-48f2-ab4b-2e5e40c2e874 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:46.344051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb512f3-2745-48f2-ab4b-2e5e40c2e874 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:46.391252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 894/1779] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:54:46.444141 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d70217-bc72-44c4-968c-ed48520c895f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:46.446206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d70217-bc72-44c4-968c-ed48520c895f service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:46.468852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 886/1780] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:54:47.499999 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6d27d-e0be-4ca7-8dc8-1d64ac0530b4 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:47.500472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6d27d-e0be-4ca7-8dc8-1d64ac0530b4 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:47.500472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6d27d-e0be-4ca7-8dc8-1d64ac0530b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:47.520734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-51c6d27d-e0be-4ca7-8dc8-1d64ac0530b4 None None] MFA Rules not processed for user `a9a3080f9aef4323952eed19ef9bb074`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:54:47.522416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-51c6d27d-e0be-4ca7-8dc8-1d64ac0530b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:54:47.596464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 895/1781] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:54:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:54:51.289407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d70217-bc72-44c4-968c-ed48520c895f service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:54:51.298802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d70217-bc72-44c4-968c-ed48520c895f service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:54:51.302503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5c119-3351-429c-b264-3b7fac29de76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:54:51.303182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5c119-3351-429c-b264-3b7fac29de76 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:54:51.303464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5c119-3351-429c-b264-3b7fac29de76 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:54:51.303745 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5c119-3351-429c-b264-3b7fac29de76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:54:51.313815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5c119-3351-429c-b264-3b7fac29de76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:54:51.315284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5c119-3351-429c-b264-3b7fac29de76 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:54:51.326258 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 887/1782] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:54:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:54:57.587210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=69,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:01.322302 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=57 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:01.376184 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d147b58-5319-43c8-8758-d97c20aac1cd None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:01.376662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d147b58-5319-43c8-8758-d97c20aac1cd None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:01.377085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d147b58-5319-43c8-8758-d97c20aac1cd None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:01.381201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 896/1783] 10.222.0.52 () {54 vars in 921 bytes} [Thu May 19 13:55:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:55:01.463624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d44f9a-f3ba-4b04-86c9-aabd1f26a7e3 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:01.463816 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d44f9a-f3ba-4b04-86c9-aabd1f26a7e3 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:01.464108 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d44f9a-f3ba-4b04-86c9-aabd1f26a7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:01.519825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-23d44f9a-f3ba-4b04-86c9-aabd1f26a7e3 None None] MFA Rules not processed for user `4f6093d0c1fc41d98c58bace81e3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:01.534844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-23d44f9a-f3ba-4b04-86c9-aabd1f26a7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:01.579807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 888/1784] 10.222.0.52 () {58 vars in 1018 bytes} [Thu May 19 13:55:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:02.021178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d147b58-5319-43c8-8758-d97c20aac1cd None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:02.023306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3d147b58-5319-43c8-8758-d97c20aac1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:02.066650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d147b58-5319-43c8-8758-d97c20aac1cd None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:02.074537 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de5864-5099-430e-aac8-b0d2ac1520fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:02.077234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de5864-5099-430e-aac8-b0d2ac1520fb service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:02.077345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de5864-5099-430e-aac8-b0d2ac1520fb service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:02.078419 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de5864-5099-430e-aac8-b0d2ac1520fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:02.097860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de5864-5099-430e-aac8-b0d2ac1520fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:02.100151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de5864-5099-430e-aac8-b0d2ac1520fb service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:02.165883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 897/1785] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:07.599147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d44f9a-f3ba-4b04-86c9-aabd1f26a7e3 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:07.601792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-23d44f9a-f3ba-4b04-86c9-aabd1f26a7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:07.691279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d44f9a-f3ba-4b04-86c9-aabd1f26a7e3 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:07.696198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:07.699045 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:07.699276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:07.699556 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] PATH_INFO: `/v3/users/f6104aa6b47f449a8c4174f1585d68ab` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:07.700872 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6104aa6b47f449a8c4174f1585d68ab)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:07.702823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:07.811363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6104aa6b47f449a8c4174f1585d68ab'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:07.827641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6104aa6b47f449a8c4174f1585d68ab'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:07.828397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6104aa6b47f449a8c4174f1585d68ab'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:07.838544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6104aa6b47f449a8c4174f1585d68ab'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:07.855576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6104aa6b47f449a8c4174f1585d68ab'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:07.862457 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 889/1786] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:07 2022] DELETE /identity/v3/users/f6104aa6b47f449a8c4174f1585d68ab => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:07.866992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94201d4c-4c80-4ac5-a3a4-512136fc7964 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:07.867473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94201d4c-4c80-4ac5-a3a4-512136fc7964 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:07.867853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94201d4c-4c80-4ac5-a3a4-512136fc7964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:07.887441 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-94201d4c-4c80-4ac5-a3a4-512136fc7964 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:07.888878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-94201d4c-4c80-4ac5-a3a4-512136fc7964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:07.962283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 898/1787] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:07.971940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:07.973498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:08.036627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d96ab1-9310-4a12-b174-c7d5e5e3bca2 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:08.040235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d878a6c-6f06-40e2-a6c0-4e91026058b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:08.040880 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d878a6c-6f06-40e2-a6c0-4e91026058b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:08.041160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d878a6c-6f06-40e2-a6c0-4e91026058b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:08.041435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d878a6c-6f06-40e2-a6c0-4e91026058b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:08.050962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d878a6c-6f06-40e2-a6c0-4e91026058b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:08.052388 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d878a6c-6f06-40e2-a6c0-4e91026058b2 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:08.062466 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 890/1788] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:08.327097 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94201d4c-4c80-4ac5-a3a4-512136fc7964 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:08.330953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-94201d4c-4c80-4ac5-a3a4-512136fc7964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:08.345353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94201d4c-4c80-4ac5-a3a4-512136fc7964 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:08.352444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:08.353319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:08.353607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:08.353888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] PATH_INFO: `/v3/projects/5bf416054d854d7fb9ce1b270eaec194` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:08.355691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bf416054d854d7fb9ce1b270eaec194)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:08.357961 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:08.425243 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bf416054d854d7fb9ce1b270eaec194', 'request_id': 'req-1996592d-8327-4c1f-bd46-e0120a4c8583'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:08.432040 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bf416054d854d7fb9ce1b270eaec194', 'request_id': 'req-1996592d-8327-4c1f-bd46-e0120a4c8583'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:08.446584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bf416054d854d7fb9ce1b270eaec194', 'request_id': 'req-1996592d-8327-4c1f-bd46-e0120a4c8583'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:08.458620 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] The token cache is being invalidate because project 5bf416054d854d7fb9ce1b270eaec194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:08.459057 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:08.460659 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:08.466942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 899/1789] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:08 2022] DELETE /identity/v3/projects/5bf416054d854d7fb9ce1b270eaec194 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:08.467740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d878a6c-6f06-40e2-a6c0-4e91026058b2 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:08.477567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d878a6c-6f06-40e2-a6c0-4e91026058b2 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:08.493974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:08.554747 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b74cf-5336-43cf-a90a-7803f7af7062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:08.555703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b74cf-5336-43cf-a90a-7803f7af7062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:08.555993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b74cf-5336-43cf-a90a-7803f7af7062 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:08.556274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b74cf-5336-43cf-a90a-7803f7af7062 admin admin] PATH_INFO: `/v3/users/746227a53b0b4a3e9cf678aef0327f50` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:08.557500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b74cf-5336-43cf-a90a-7803f7af7062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=746227a53b0b4a3e9cf678aef0327f50)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:08.560190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b74cf-5336-43cf-a90a-7803f7af7062 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:08.622307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-321b74cf-5336-43cf-a90a-7803f7af7062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746227a53b0b4a3e9cf678aef0327f50'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:08.630534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-321b74cf-5336-43cf-a90a-7803f7af7062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746227a53b0b4a3e9cf678aef0327f50'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:08.631331 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-321b74cf-5336-43cf-a90a-7803f7af7062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746227a53b0b4a3e9cf678aef0327f50'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:08.643216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-321b74cf-5336-43cf-a90a-7803f7af7062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746227a53b0b4a3e9cf678aef0327f50'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:08.662151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-321b74cf-5336-43cf-a90a-7803f7af7062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746227a53b0b4a3e9cf678aef0327f50'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:08.671962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 891/1790] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:08 2022] DELETE /identity/v3/users/746227a53b0b4a3e9cf678aef0327f50 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:08.991258 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:08.992708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:09.008279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996592d-8327-4c1f-bd46-e0120a4c8583 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:09.083182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11bcdd-0539-471d-ae48-429cf44c52ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:09.084047 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11bcdd-0539-471d-ae48-429cf44c52ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:09.084332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11bcdd-0539-471d-ae48-429cf44c52ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:09.084610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11bcdd-0539-471d-ae48-429cf44c52ea admin admin] PATH_INFO: `/v3/projects/b27597fc470c4dba8ccb8d8a8d67b404` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:09.085788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11bcdd-0539-471d-ae48-429cf44c52ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27597fc470c4dba8ccb8d8a8d67b404)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:09.087728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11bcdd-0539-471d-ae48-429cf44c52ea admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:09.099692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:09.137130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-6c11bcdd-0539-471d-ae48-429cf44c52ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27597fc470c4dba8ccb8d8a8d67b404', 'request_id': 'req-6c11bcdd-0539-471d-ae48-429cf44c52ea'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:09.147577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-6c11bcdd-0539-471d-ae48-429cf44c52ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27597fc470c4dba8ccb8d8a8d67b404', 'request_id': 'req-6c11bcdd-0539-471d-ae48-429cf44c52ea'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:09.157105 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-6c11bcdd-0539-471d-ae48-429cf44c52ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27597fc470c4dba8ccb8d8a8d67b404', 'request_id': 'req-6c11bcdd-0539-471d-ae48-429cf44c52ea'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:09.168692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-6c11bcdd-0539-471d-ae48-429cf44c52ea admin admin] The token cache is being invalidate because project b27597fc470c4dba8ccb8d8a8d67b404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:09.169009 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-6c11bcdd-0539-471d-ae48-429cf44c52ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:09.169646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-6c11bcdd-0539-471d-ae48-429cf44c52ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:09.171367 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 900/1791] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:08 2022] DELETE /identity/v3/projects/b27597fc470c4dba8ccb8d8a8d67b404 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:09.176330 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813233a-9b81-402c-8252-edec82496f04 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:09.176624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813233a-9b81-402c-8252-edec82496f04 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:09.176907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813233a-9b81-402c-8252-edec82496f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:09.196498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3813233a-9b81-402c-8252-edec82496f04 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:09.197755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3813233a-9b81-402c-8252-edec82496f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:09.271669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 892/1792] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:09.278833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab477f10-8563-4d46-9bc4-e2d2e359e00e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:09.279221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab477f10-8563-4d46-9bc4-e2d2e359e00e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:09.279572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab477f10-8563-4d46-9bc4-e2d2e359e00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:09.298843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ab477f10-8563-4d46-9bc4-e2d2e359e00e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:09.300086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ab477f10-8563-4d46-9bc4-e2d2e359e00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:09.308097 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 901/1793] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:09.312942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813233a-9b81-402c-8252-edec82496f04 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:09.322464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813233a-9b81-402c-8252-edec82496f04 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:09.327907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf630d-fe88-454e-8911-2401f539bf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:09.328570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf630d-fe88-454e-8911-2401f539bf99 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:09.328861 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf630d-fe88-454e-8911-2401f539bf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:09.329139 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf630d-fe88-454e-8911-2401f539bf99 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:09.329865 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf630d-fe88-454e-8911-2401f539bf99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:09.330175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf630d-fe88-454e-8911-2401f539bf99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:09.331803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf630d-fe88-454e-8911-2401f539bf99 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:09.343639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 893/1794] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:55:09.347954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab477f10-8563-4d46-9bc4-e2d2e359e00e None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:09.357293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab477f10-8563-4d46-9bc4-e2d2e359e00e None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:09.362198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d326b-26e7-4e29-a8e3-157b01733f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:09.362854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d326b-26e7-4e29-a8e3-157b01733f41 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:09.363173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d326b-26e7-4e29-a8e3-157b01733f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:09.363456 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d326b-26e7-4e29-a8e3-157b01733f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:09.364165 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d326b-26e7-4e29-a8e3-157b01733f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:09.366083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d326b-26e7-4e29-a8e3-157b01733f41 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:09.396124 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 902/1795] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:09 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:09.400381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf630d-fe88-454e-8911-2401f539bf99 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:09.411308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf630d-fe88-454e-8911-2401f539bf99 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:09.436057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:09.486309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a960b3-9bd2-4cf5-a384-ccf544ae1152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:09.487045 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a960b3-9bd2-4cf5-a384-ccf544ae1152 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:09.487334 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a960b3-9bd2-4cf5-a384-ccf544ae1152 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:09.487613 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a960b3-9bd2-4cf5-a384-ccf544ae1152 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:09.488335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a960b3-9bd2-4cf5-a384-ccf544ae1152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:09.490357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a960b3-9bd2-4cf5-a384-ccf544ae1152 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:09.511338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 894/1796] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:09 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:55:09.516445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d326b-26e7-4e29-a8e3-157b01733f41 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:09.526775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d326b-26e7-4e29-a8e3-157b01733f41 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:09.532076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6f34e-76cd-4f2f-82f9-a07506e9add5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:09.532777 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6f34e-76cd-4f2f-82f9-a07506e9add5 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:09.533062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6f34e-76cd-4f2f-82f9-a07506e9add5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:09.533578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6f34e-76cd-4f2f-82f9-a07506e9add5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:09.534505 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6f34e-76cd-4f2f-82f9-a07506e9add5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:09.536183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6f34e-76cd-4f2f-82f9-a07506e9add5 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:09.548324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 903/1797] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:09.552413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a960b3-9bd2-4cf5-a384-ccf544ae1152 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:09.564086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a960b3-9bd2-4cf5-a384-ccf544ae1152 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:09.568855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d151335-d594-4846-b756-75ac54f406f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:09.569537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d151335-d594-4846-b756-75ac54f406f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:09.569852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d151335-d594-4846-b756-75ac54f406f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:09.570125 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d151335-d594-4846-b756-75ac54f406f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:09.576071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d151335-d594-4846-b756-75ac54f406f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:09.577833 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d151335-d594-4846-b756-75ac54f406f2 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:09.589826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 895/1798] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:09.594250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6f34e-76cd-4f2f-82f9-a07506e9add5 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:09.604538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6f34e-76cd-4f2f-82f9-a07506e9add5 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:09.610324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec07781-42d6-4956-a915-f8f5ed2593ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:09.611278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec07781-42d6-4956-a915-f8f5ed2593ed admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:09.611746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec07781-42d6-4956-a915-f8f5ed2593ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:09.612024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec07781-42d6-4956-a915-f8f5ed2593ed admin admin] PATH_INFO: `/v3/projects/a4f79bafb3b84c7b838536ead7a8a365/users/2928a6e38d0b4f0e84c604bff031c4ea/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:09.630267 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec07781-42d6-4956-a915-f8f5ed2593ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f79bafb3b84c7b838536ead7a8a365, user_id=2928a6e38d0b4f0e84c604bff031c4ea, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:09.632957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec07781-42d6-4956-a915-f8f5ed2593ed admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:09.647967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 904/1799] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:09 2022] PUT /identity/v3/projects/a4f79bafb3b84c7b838536ead7a8a365/users/2928a6e38d0b4f0e84c604bff031c4ea/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:09.658596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb39a2a-6835-4616-8237-d3ea346a9e30 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:09.659082 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb39a2a-6835-4616-8237-d3ea346a9e30 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:09.659265 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb39a2a-6835-4616-8237-d3ea346a9e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:09.689562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6fb39a2a-6835-4616-8237-d3ea346a9e30 None None] MFA Rules not processed for user `2928a6e38d0b4f0e84c604bff031c4ea`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:09.690779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6fb39a2a-6835-4616-8237-d3ea346a9e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:09.758134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 896/1800] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:09 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:09.762552 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec07781-42d6-4956-a915-f8f5ed2593ed admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:09.825649 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec07781-42d6-4956-a915-f8f5ed2593ed admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:09.830395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c8958-3d43-43f6-847b-7937c687a7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:09.831086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c8958-3d43-43f6-847b-7937c687a7dc admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:09.831376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c8958-3d43-43f6-847b-7937c687a7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:09.831651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c8958-3d43-43f6-847b-7937c687a7dc admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:09.832351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c8958-3d43-43f6-847b-7937c687a7dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:09.834246 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c8958-3d43-43f6-847b-7937c687a7dc admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:09.861938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 905/1801] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:09 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:09.866965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb39a2a-6835-4616-8237-d3ea346a9e30 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:09.878986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb39a2a-6835-4616-8237-d3ea346a9e30 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:09.940606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9b43b-a397-4036-ba3b-aaf807741cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:09.941292 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b43b-a397-4036-ba3b-aaf807741cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:09.941620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b43b-a397-4036-ba3b-aaf807741cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:09.941900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b43b-a397-4036-ba3b-aaf807741cd1 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:09.942610 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9b43b-a397-4036-ba3b-aaf807741cd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:09.944623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9b43b-a397-4036-ba3b-aaf807741cd1 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:09.969078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 897/1802] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:09 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:09.972888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c8958-3d43-43f6-847b-7937c687a7dc admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:09.984318 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c8958-3d43-43f6-847b-7937c687a7dc admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:09.990414 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b5175-7748-4a55-8abb-047fdc820d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:09.991016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b5175-7748-4a55-8abb-047fdc820d4e admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:09.991304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b5175-7748-4a55-8abb-047fdc820d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:09.991859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b5175-7748-4a55-8abb-047fdc820d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:09.992310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b5175-7748-4a55-8abb-047fdc820d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:09.993908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b5175-7748-4a55-8abb-047fdc820d4e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:10.006893 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 906/1803] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:10.010204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9b43b-a397-4036-ba3b-aaf807741cd1 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:10.020854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9b43b-a397-4036-ba3b-aaf807741cd1 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:10.025602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c725f25-0937-4b72-a7f0-061cd8d64bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:10.026503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c725f25-0937-4b72-a7f0-061cd8d64bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:10.026791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c725f25-0937-4b72-a7f0-061cd8d64bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:10.027113 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c725f25-0937-4b72-a7f0-061cd8d64bd2 admin admin] PATH_INFO: `/v3/projects/9da16f96471d44a99d902b5a65ff6ced/users/3add8e6e292a4d7b8a6883430f25f42e/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:10.044383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c725f25-0937-4b72-a7f0-061cd8d64bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9da16f96471d44a99d902b5a65ff6ced, user_id=3add8e6e292a4d7b8a6883430f25f42e, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:10.047200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c725f25-0937-4b72-a7f0-061cd8d64bd2 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:10.050411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:10.063705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 898/1804] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:10 2022] PUT /identity/v3/projects/9da16f96471d44a99d902b5a65ff6ced/users/3add8e6e292a4d7b8a6883430f25f42e/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:10.067658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b5175-7748-4a55-8abb-047fdc820d4e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:10.079049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b5175-7748-4a55-8abb-047fdc820d4e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:10.087975 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:10.139903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b949ec8-b1c6-4853-9f6c-460c5c7e92e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:10.140571 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b949ec8-b1c6-4853-9f6c-460c5c7e92e5 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:10.140854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b949ec8-b1c6-4853-9f6c-460c5c7e92e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:10.141124 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b949ec8-b1c6-4853-9f6c-460c5c7e92e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:10.141846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b949ec8-b1c6-4853-9f6c-460c5c7e92e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:10.143485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b949ec8-b1c6-4853-9f6c-460c5c7e92e5 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:10.152964 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 907/1805] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:10.157058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c725f25-0937-4b72-a7f0-061cd8d64bd2 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:10.167220 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c725f25-0937-4b72-a7f0-061cd8d64bd2 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:10.171780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526dd53-91fe-4ee7-a620-ea88a731e47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:10.172652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526dd53-91fe-4ee7-a620-ea88a731e47b admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:10.172929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526dd53-91fe-4ee7-a620-ea88a731e47b admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:10.173202 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526dd53-91fe-4ee7-a620-ea88a731e47b admin admin] PATH_INFO: `/v3/projects/9da16f96471d44a99d902b5a65ff6ced/users/3add8e6e292a4d7b8a6883430f25f42e/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:10.175170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526dd53-91fe-4ee7-a620-ea88a731e47b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9da16f96471d44a99d902b5a65ff6ced, user_id=3add8e6e292a4d7b8a6883430f25f42e, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:10.177776 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526dd53-91fe-4ee7-a620-ea88a731e47b admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:10.188427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 899/1806] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:10 2022] PUT /identity/v3/projects/9da16f96471d44a99d902b5a65ff6ced/users/3add8e6e292a4d7b8a6883430f25f42e/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:10.199724 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b743308-5422-4abe-8d48-22e1c9b8b19d None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:10.200086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b743308-5422-4abe-8d48-22e1c9b8b19d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:10.201077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b743308-5422-4abe-8d48-22e1c9b8b19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:10.226615 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-5b743308-5422-4abe-8d48-22e1c9b8b19d None None] MFA Rules not processed for user `3add8e6e292a4d7b8a6883430f25f42e`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:10.227852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5b743308-5422-4abe-8d48-22e1c9b8b19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:10.299420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 908/1807] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:10 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:10.311576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526dd53-91fe-4ee7-a620-ea88a731e47b admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:10.313356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c526dd53-91fe-4ee7-a620-ea88a731e47b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:10.376460 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526dd53-91fe-4ee7-a620-ea88a731e47b admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:10.380592 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22017b-e442-4ba7-a1b0-3576613cdd60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:10.381288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22017b-e442-4ba7-a1b0-3576613cdd60 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:10.381571 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22017b-e442-4ba7-a1b0-3576613cdd60 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:10.381847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22017b-e442-4ba7-a1b0-3576613cdd60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:10.393207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22017b-e442-4ba7-a1b0-3576613cdd60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:10.399261 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22017b-e442-4ba7-a1b0-3576613cdd60 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:10.406738 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 900/1808] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:10 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:11.912560 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b743308-5422-4abe-8d48-22e1c9b8b19d None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:11.987877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b743308-5422-4abe-8d48-22e1c9b8b19d None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:11.993820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:11.995122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:11.995552 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:11.996005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] PATH_INFO: `/v3/users/2928a6e38d0b4f0e84c604bff031c4ea` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:11.998021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2928a6e38d0b4f0e84c604bff031c4ea)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:12.000869 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:12.070077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:12.085998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2928a6e38d0b4f0e84c604bff031c4ea'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.134387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2928a6e38d0b4f0e84c604bff031c4ea'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.147534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2928a6e38d0b4f0e84c604bff031c4ea'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.148418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2928a6e38d0b4f0e84c604bff031c4ea'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.149083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2928a6e38d0b4f0e84c604bff031c4ea'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.180283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 909/1809] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:11 2022] DELETE /identity/v3/users/2928a6e38d0b4f0e84c604bff031c4ea => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:12.185648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c99eb6-2b1b-4991-8452-5a0c3085ca84 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:12.186119 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c99eb6-2b1b-4991-8452-5a0c3085ca84 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:12.186565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c99eb6-2b1b-4991-8452-5a0c3085ca84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:12.214194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-f1c99eb6-2b1b-4991-8452-5a0c3085ca84 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:12.216751 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f1c99eb6-2b1b-4991-8452-5a0c3085ca84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:12.308613 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 901/1810] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:12.318972 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:12.322236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:12.407398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfcf11-a93f-499a-9680-fa62f4df2a74 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:12.411019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0736c-7e2a-4564-bc65-bfc78eff885c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:12.411749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0736c-7e2a-4564-bc65-bfc78eff885c service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:12.411939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0736c-7e2a-4564-bc65-bfc78eff885c service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:12.412212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0736c-7e2a-4564-bc65-bfc78eff885c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:12.422285 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0736c-7e2a-4564-bc65-bfc78eff885c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:12.423813 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0736c-7e2a-4564-bc65-bfc78eff885c service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:12.434003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 910/1811] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:12.737098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c99eb6-2b1b-4991-8452-5a0c3085ca84 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:12.738540 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f1c99eb6-2b1b-4991-8452-5a0c3085ca84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:12.751842 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c99eb6-2b1b-4991-8452-5a0c3085ca84 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:12.757300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:12.758223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:12.758466 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:12.758749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] PATH_INFO: `/v3/projects/a4f79bafb3b84c7b838536ead7a8a365` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:12.760072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f79bafb3b84c7b838536ead7a8a365)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:12.761980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:12.818683 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f79bafb3b84c7b838536ead7a8a365', 'request_id': 'req-3dbb9702-3767-4bf9-b001-5006f36b3748'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.825663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f79bafb3b84c7b838536ead7a8a365', 'request_id': 'req-3dbb9702-3767-4bf9-b001-5006f36b3748'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.835731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f79bafb3b84c7b838536ead7a8a365', 'request_id': 'req-3dbb9702-3767-4bf9-b001-5006f36b3748'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.847039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] The token cache is being invalidate because project a4f79bafb3b84c7b838536ead7a8a365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:12.847331 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.848082 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.849627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 902/1812] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:12 2022] DELETE /identity/v3/projects/a4f79bafb3b84c7b838536ead7a8a365 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:12.853641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0736c-7e2a-4564-bc65-bfc78eff885c service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:12.864295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0736c-7e2a-4564-bc65-bfc78eff885c service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:12.923397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61869866-cab5-4e95-8170-902acf520c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:12.924637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61869866-cab5-4e95-8170-902acf520c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:12.925026 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61869866-cab5-4e95-8170-902acf520c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:12.925400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61869866-cab5-4e95-8170-902acf520c6a admin admin] PATH_INFO: `/v3/users/3add8e6e292a4d7b8a6883430f25f42e` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:12.926901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61869866-cab5-4e95-8170-902acf520c6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3add8e6e292a4d7b8a6883430f25f42e)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:12.929643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61869866-cab5-4e95-8170-902acf520c6a admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:12.974101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-61869866-cab5-4e95-8170-902acf520c6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3add8e6e292a4d7b8a6883430f25f42e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.982767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-61869866-cab5-4e95-8170-902acf520c6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3add8e6e292a4d7b8a6883430f25f42e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.985341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:12.992880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-61869866-cab5-4e95-8170-902acf520c6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3add8e6e292a4d7b8a6883430f25f42e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.994118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-61869866-cab5-4e95-8170-902acf520c6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3add8e6e292a4d7b8a6883430f25f42e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:12.995082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-61869866-cab5-4e95-8170-902acf520c6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3add8e6e292a4d7b8a6883430f25f42e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:13.013266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 911/1813] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:12 2022] DELETE /identity/v3/users/3add8e6e292a4d7b8a6883430f25f42e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:13.282217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:13.283744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:13.297083 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb9702-3767-4bf9-b001-5006f36b3748 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:13.302727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:13.362341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64121e7-98c3-443e-a7d4-9fff8f90058a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:13.363326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64121e7-98c3-443e-a7d4-9fff8f90058a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:13.363635 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64121e7-98c3-443e-a7d4-9fff8f90058a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:13.364048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64121e7-98c3-443e-a7d4-9fff8f90058a admin admin] PATH_INFO: `/v3/projects/9da16f96471d44a99d902b5a65ff6ced` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:13.365257 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64121e7-98c3-443e-a7d4-9fff8f90058a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9da16f96471d44a99d902b5a65ff6ced)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:13.367201 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64121e7-98c3-443e-a7d4-9fff8f90058a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:13.429537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d64121e7-98c3-443e-a7d4-9fff8f90058a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9da16f96471d44a99d902b5a65ff6ced', 'request_id': 'req-d64121e7-98c3-443e-a7d4-9fff8f90058a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:13.430455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d64121e7-98c3-443e-a7d4-9fff8f90058a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9da16f96471d44a99d902b5a65ff6ced', 'request_id': 'req-d64121e7-98c3-443e-a7d4-9fff8f90058a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:13.439394 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d64121e7-98c3-443e-a7d4-9fff8f90058a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9da16f96471d44a99d902b5a65ff6ced', 'request_id': 'req-d64121e7-98c3-443e-a7d4-9fff8f90058a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:13.450657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d64121e7-98c3-443e-a7d4-9fff8f90058a admin admin] The token cache is being invalidate because project 9da16f96471d44a99d902b5a65ff6ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:13.451053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d64121e7-98c3-443e-a7d4-9fff8f90058a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:13.451790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d64121e7-98c3-443e-a7d4-9fff8f90058a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:13.453380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 903/1814] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:13 2022] DELETE /identity/v3/projects/9da16f96471d44a99d902b5a65ff6ced => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:13.458452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9e4f6-bb0c-4a43-b8a6-883f2303b7b3 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:13.458739 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9e4f6-bb0c-4a43-b8a6-883f2303b7b3 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:13.459049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9e4f6-bb0c-4a43-b8a6-883f2303b7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:13.487088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f7d9e4f6-bb0c-4a43-b8a6-883f2303b7b3 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:13.488523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f7d9e4f6-bb0c-4a43-b8a6-883f2303b7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:13.576814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 912/1815] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:13.579406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fd393-dde1-49b8-8994-2fd2e9845a44 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:13.579660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fd393-dde1-49b8-8994-2fd2e9845a44 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:13.579971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fd393-dde1-49b8-8994-2fd2e9845a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:13.636519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3c0fd393-dde1-49b8-8994-2fd2e9845a44 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:13.638450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3c0fd393-dde1-49b8-8994-2fd2e9845a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:13.650502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 904/1816] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:13.655581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9e4f6-bb0c-4a43-b8a6-883f2303b7b3 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:13.665473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9e4f6-bb0c-4a43-b8a6-883f2303b7b3 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:13.670284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e5c99-887d-471c-9a74-40783c96cb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:13.670969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e5c99-887d-471c-9a74-40783c96cb1f admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:13.671257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e5c99-887d-471c-9a74-40783c96cb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:13.671525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e5c99-887d-471c-9a74-40783c96cb1f admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:13.672254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e5c99-887d-471c-9a74-40783c96cb1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:13.672558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e5c99-887d-471c-9a74-40783c96cb1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:13.674148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e5c99-887d-471c-9a74-40783c96cb1f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:13.688357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 913/1817] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:55:13.695855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fd393-dde1-49b8-8994-2fd2e9845a44 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:13.715996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fd393-dde1-49b8-8994-2fd2e9845a44 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:13.721582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804a6ab-72fd-4b56-8e99-b8816dc0c3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:13.722374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804a6ab-72fd-4b56-8e99-b8816dc0c3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:13.722679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804a6ab-72fd-4b56-8e99-b8816dc0c3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:13.723027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804a6ab-72fd-4b56-8e99-b8816dc0c3c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:13.723721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804a6ab-72fd-4b56-8e99-b8816dc0c3c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:13.725666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804a6ab-72fd-4b56-8e99-b8816dc0c3c2 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:13.755415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 905/1818] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:13 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:13.759638 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e5c99-887d-471c-9a74-40783c96cb1f admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:13.771662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e5c99-887d-471c-9a74-40783c96cb1f admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:13.835711 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1340d-9a0a-427c-b1ce-bf3320a3392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:13.836392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1340d-9a0a-427c-b1ce-bf3320a3392a admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:13.836729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1340d-9a0a-427c-b1ce-bf3320a3392a admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:13.837016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1340d-9a0a-427c-b1ce-bf3320a3392a admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:13.837822 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1340d-9a0a-427c-b1ce-bf3320a3392a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:13.839903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1340d-9a0a-427c-b1ce-bf3320a3392a admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:13.860579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 914/1819] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:13 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:13.864854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804a6ab-72fd-4b56-8e99-b8816dc0c3c2 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:13.875910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804a6ab-72fd-4b56-8e99-b8816dc0c3c2 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:13.881541 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38875470-571d-436f-83e5-161c45277785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:13.882299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38875470-571d-436f-83e5-161c45277785 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:13.882588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38875470-571d-436f-83e5-161c45277785 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:13.883056 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38875470-571d-436f-83e5-161c45277785 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:13.883831 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38875470-571d-436f-83e5-161c45277785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:13.885428 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38875470-571d-436f-83e5-161c45277785 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:13.900009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 906/1820] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:13.901175 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1340d-9a0a-427c-b1ce-bf3320a3392a admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:13.912544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1340d-9a0a-427c-b1ce-bf3320a3392a admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:13.930714 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca15b49-5e10-4532-9464-0ca186f7a958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:13.931472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca15b49-5e10-4532-9464-0ca186f7a958 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:13.940619 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca15b49-5e10-4532-9464-0ca186f7a958 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:13.940619 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca15b49-5e10-4532-9464-0ca186f7a958 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:13.940619 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca15b49-5e10-4532-9464-0ca186f7a958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:13.940619 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca15b49-5e10-4532-9464-0ca186f7a958 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:13.946952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 915/1821] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:13.950115 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38875470-571d-436f-83e5-161c45277785 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:13.960831 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38875470-571d-436f-83e5-161c45277785 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:13.965432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dfee4-0b60-450c-9931-a40e57b67e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:13.966956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dfee4-0b60-450c-9931-a40e57b67e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:13.967353 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dfee4-0b60-450c-9931-a40e57b67e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:13.967634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dfee4-0b60-450c-9931-a40e57b67e92 admin admin] PATH_INFO: `/v3/projects/66591b8712794b89b55fd612be386c19/users/e0bb9e6a17484f4aa93ed29aec64781c/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:13.982822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42dfee4-0b60-450c-9931-a40e57b67e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66591b8712794b89b55fd612be386c19, user_id=e0bb9e6a17484f4aa93ed29aec64781c, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:13.985519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42dfee4-0b60-450c-9931-a40e57b67e92 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:13.999414 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 907/1822] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:13 2022] PUT /identity/v3/projects/66591b8712794b89b55fd612be386c19/users/e0bb9e6a17484f4aa93ed29aec64781c/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:14.010026 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b0582-15bc-4515-837a-d8d73dfc896a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:14.010350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b0582-15bc-4515-837a-d8d73dfc896a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:14.010685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b0582-15bc-4515-837a-d8d73dfc896a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:14.039296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f51b0582-15bc-4515-837a-d8d73dfc896a None None] MFA Rules not processed for user `e0bb9e6a17484f4aa93ed29aec64781c`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:14.040516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f51b0582-15bc-4515-837a-d8d73dfc896a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:14.070823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:14.111613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 916/1823] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:14.115973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dfee4-0b60-450c-9931-a40e57b67e92 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:14.155727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:14.179886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dfee4-0b60-450c-9931-a40e57b67e92 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:14.185124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4a2e1-4d1f-42b5-9783-f1e95a6a93d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:14.185783 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4a2e1-4d1f-42b5-9783-f1e95a6a93d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:14.186062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4a2e1-4d1f-42b5-9783-f1e95a6a93d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:14.186338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4a2e1-4d1f-42b5-9783-f1e95a6a93d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:14.187067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4a2e1-4d1f-42b5-9783-f1e95a6a93d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:14.188944 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4a2e1-4d1f-42b5-9783-f1e95a6a93d8 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:14.217206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 908/1824] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:14 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:14.221230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b0582-15bc-4515-837a-d8d73dfc896a None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:14.231773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b0582-15bc-4515-837a-d8d73dfc896a None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:14.286656 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77c55f-d9ff-48e2-95cb-d3a25d6e6f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:14.287383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77c55f-d9ff-48e2-95cb-d3a25d6e6f03 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:14.287668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77c55f-d9ff-48e2-95cb-d3a25d6e6f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:14.287945 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77c55f-d9ff-48e2-95cb-d3a25d6e6f03 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:14.288675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77c55f-d9ff-48e2-95cb-d3a25d6e6f03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:14.290614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77c55f-d9ff-48e2-95cb-d3a25d6e6f03 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:14.309858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 917/1825] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:14 2022] POST /identity/v3/users => generated 291 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:14.314077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4a2e1-4d1f-42b5-9783-f1e95a6a93d8 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:14.324077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4a2e1-4d1f-42b5-9783-f1e95a6a93d8 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:14.329376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dce43-f976-4b0e-83ad-902c7072f2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:14.330019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dce43-f976-4b0e-83ad-902c7072f2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:14.330299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dce43-f976-4b0e-83ad-902c7072f2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:14.330574 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dce43-f976-4b0e-83ad-902c7072f2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:14.331328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099dce43-f976-4b0e-83ad-902c7072f2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:14.332899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099dce43-f976-4b0e-83ad-902c7072f2c2 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:14.343976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 909/1826] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:14.348002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77c55f-d9ff-48e2-95cb-d3a25d6e6f03 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:14.358006 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77c55f-d9ff-48e2-95cb-d3a25d6e6f03 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:14.362560 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd19bb-bd73-44dc-8631-889ebef8cf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:14.363457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd19bb-bd73-44dc-8631-889ebef8cf1a admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:14.363738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd19bb-bd73-44dc-8631-889ebef8cf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:14.364011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd19bb-bd73-44dc-8631-889ebef8cf1a admin admin] PATH_INFO: `/v3/projects/69f59db0b6ed463f8a814400e287bac9/users/9f0953c56ad94d848e9963c44200bc95/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:14.378790 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd19bb-bd73-44dc-8631-889ebef8cf1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=69f59db0b6ed463f8a814400e287bac9, user_id=9f0953c56ad94d848e9963c44200bc95, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:14.381423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd19bb-bd73-44dc-8631-889ebef8cf1a admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:14.395515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 918/1827] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:14 2022] PUT /identity/v3/projects/69f59db0b6ed463f8a814400e287bac9/users/9f0953c56ad94d848e9963c44200bc95/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:14.399543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dce43-f976-4b0e-83ad-902c7072f2c2 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:14.410642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dce43-f976-4b0e-83ad-902c7072f2c2 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:14.468928 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d5628-5d0b-440e-aa7e-e720c83a0bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:14.469590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d5628-5d0b-440e-aa7e-e720c83a0bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:14.469922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d5628-5d0b-440e-aa7e-e720c83a0bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:14.470196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d5628-5d0b-440e-aa7e-e720c83a0bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:14.470942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d5628-5d0b-440e-aa7e-e720c83a0bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:14.472589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d5628-5d0b-440e-aa7e-e720c83a0bb1 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:14.483179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 910/1828] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:14.487552 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd19bb-bd73-44dc-8631-889ebef8cf1a admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:14.498617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd19bb-bd73-44dc-8631-889ebef8cf1a admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:14.503257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f858b6-1cc3-4acc-afb5-2bcdebe20f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:14.504376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f858b6-1cc3-4acc-afb5-2bcdebe20f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:14.504661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f858b6-1cc3-4acc-afb5-2bcdebe20f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:14.504936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f858b6-1cc3-4acc-afb5-2bcdebe20f6f admin admin] PATH_INFO: `/v3/projects/69f59db0b6ed463f8a814400e287bac9/users/9f0953c56ad94d848e9963c44200bc95/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:14.506929 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f858b6-1cc3-4acc-afb5-2bcdebe20f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=69f59db0b6ed463f8a814400e287bac9, user_id=9f0953c56ad94d848e9963c44200bc95, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:14.509566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f858b6-1cc3-4acc-afb5-2bcdebe20f6f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:14.520580 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 919/1829] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:14 2022] PUT /identity/v3/projects/69f59db0b6ed463f8a814400e287bac9/users/9f0953c56ad94d848e9963c44200bc95/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:14.532021 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c584a05-0a60-4056-8d6b-9d42a4fe5361 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:14.532407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c584a05-0a60-4056-8d6b-9d42a4fe5361 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:14.532689 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c584a05-0a60-4056-8d6b-9d42a4fe5361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:14.558024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-2c584a05-0a60-4056-8d6b-9d42a4fe5361 None None] MFA Rules not processed for user `9f0953c56ad94d848e9963c44200bc95`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:14.559280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2c584a05-0a60-4056-8d6b-9d42a4fe5361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:14.625681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 911/1830] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:14 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:14.637264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f858b6-1cc3-4acc-afb5-2bcdebe20f6f admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:14.639686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-48f858b6-1cc3-4acc-afb5-2bcdebe20f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:14.713289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f858b6-1cc3-4acc-afb5-2bcdebe20f6f admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:14.717765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc347bb2-c438-48c9-b3de-ec45a9afaeef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:14.718040 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc347bb2-c438-48c9-b3de-ec45a9afaeef service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:14.718325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc347bb2-c438-48c9-b3de-ec45a9afaeef service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:14.718743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc347bb2-c438-48c9-b3de-ec45a9afaeef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:14.729241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc347bb2-c438-48c9-b3de-ec45a9afaeef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:14.731020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc347bb2-c438-48c9-b3de-ec45a9afaeef service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:14.740661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 920/1831] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:14 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:14.789221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c584a05-0a60-4056-8d6b-9d42a4fe5361 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:14.790608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2c584a05-0a60-4056-8d6b-9d42a4fe5361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:14.803008 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c584a05-0a60-4056-8d6b-9d42a4fe5361 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:14.806606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690603a-f7a0-4163-b461-4a03e2bac3a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:14.807297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690603a-f7a0-4163-b461-4a03e2bac3a5 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:14.807574 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690603a-f7a0-4163-b461-4a03e2bac3a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:14.807847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690603a-f7a0-4163-b461-4a03e2bac3a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:14.845530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:14.876047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690603a-f7a0-4163-b461-4a03e2bac3a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:14.877481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690603a-f7a0-4163-b461-4a03e2bac3a5 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:14.887186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 912/1832] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:18.274081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc347bb2-c438-48c9-b3de-ec45a9afaeef service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:18.300851 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:18.340771 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc347bb2-c438-48c9-b3de-ec45a9afaeef service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:18.345235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:18.346111 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:18.346425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:18.346705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] PATH_INFO: `/v3/users/e0bb9e6a17484f4aa93ed29aec64781c` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:18.347942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0bb9e6a17484f4aa93ed29aec64781c)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:18.349881 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:18.396272 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0bb9e6a17484f4aa93ed29aec64781c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:18.405462 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0bb9e6a17484f4aa93ed29aec64781c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:18.412855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0bb9e6a17484f4aa93ed29aec64781c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:18.413501 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0bb9e6a17484f4aa93ed29aec64781c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:18.414119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0bb9e6a17484f4aa93ed29aec64781c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:18.435095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 921/1833] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:18 2022] DELETE /identity/v3/users/e0bb9e6a17484f4aa93ed29aec64781c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:18.441064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7d2d8-a5a4-4a50-8736-dde67e2f7bba None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:18.441522 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7d2d8-a5a4-4a50-8736-dde67e2f7bba None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:18.441818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7d2d8-a5a4-4a50-8736-dde67e2f7bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:18.460307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a1b7d2d8-a5a4-4a50-8736-dde67e2f7bba None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:18.461841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a1b7d2d8-a5a4-4a50-8736-dde67e2f7bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:18.521623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 913/1834] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:18.532286 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:18.533819 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:18.595133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7396ae-7ae1-443b-94a4-2e7fffccc228 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:18.598907 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31561f3a-b3a9-4a8f-8362-cb37bd192ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:18.599605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31561f3a-b3a9-4a8f-8362-cb37bd192ded service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:18.599878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31561f3a-b3a9-4a8f-8362-cb37bd192ded service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:18.600158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31561f3a-b3a9-4a8f-8362-cb37bd192ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:18.610696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31561f3a-b3a9-4a8f-8362-cb37bd192ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:18.612201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31561f3a-b3a9-4a8f-8362-cb37bd192ded service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:18.627903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 922/1835] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:19.020580 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7d2d8-a5a4-4a50-8736-dde67e2f7bba None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:19.022567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a1b7d2d8-a5a4-4a50-8736-dde67e2f7bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:19.052531 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7d2d8-a5a4-4a50-8736-dde67e2f7bba None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:19.066134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:19.067215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:19.067511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:19.067789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] PATH_INFO: `/v3/projects/66591b8712794b89b55fd612be386c19` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:19.071953 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66591b8712794b89b55fd612be386c19)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:19.079701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:19.202685 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66591b8712794b89b55fd612be386c19', 'request_id': 'req-765c7d30-3295-4b17-aa49-6fe2878087a0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:19.215547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66591b8712794b89b55fd612be386c19', 'request_id': 'req-765c7d30-3295-4b17-aa49-6fe2878087a0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:19.242420 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66591b8712794b89b55fd612be386c19', 'request_id': 'req-765c7d30-3295-4b17-aa49-6fe2878087a0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:19.274383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] The token cache is being invalidate because project 66591b8712794b89b55fd612be386c19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:19.274706 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:19.280147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:19.283204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 914/1836] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:19 2022] DELETE /identity/v3/projects/66591b8712794b89b55fd612be386c19 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:19.287334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31561f3a-b3a9-4a8f-8362-cb37bd192ded service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:19.321854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31561f3a-b3a9-4a8f-8362-cb37bd192ded service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:19.433317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:19.451645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d0b6d-ad95-4aa1-a9ae-3322f7126d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:19.452470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d0b6d-ad95-4aa1-a9ae-3322f7126d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:19.453075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d0b6d-ad95-4aa1-a9ae-3322f7126d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:19.453601 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d0b6d-ad95-4aa1-a9ae-3322f7126d02 admin admin] PATH_INFO: `/v3/users/9f0953c56ad94d848e9963c44200bc95` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:19.455426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d0b6d-ad95-4aa1-a9ae-3322f7126d02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f0953c56ad94d848e9963c44200bc95)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:19.457368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d0b6d-ad95-4aa1-a9ae-3322f7126d02 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:19.519225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-af4d0b6d-ad95-4aa1-a9ae-3322f7126d02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f0953c56ad94d848e9963c44200bc95'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:19.529632 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-af4d0b6d-ad95-4aa1-a9ae-3322f7126d02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f0953c56ad94d848e9963c44200bc95'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:19.539245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-af4d0b6d-ad95-4aa1-a9ae-3322f7126d02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f0953c56ad94d848e9963c44200bc95'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:19.540444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-af4d0b6d-ad95-4aa1-a9ae-3322f7126d02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f0953c56ad94d848e9963c44200bc95'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:19.540947 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-af4d0b6d-ad95-4aa1-a9ae-3322f7126d02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f0953c56ad94d848e9963c44200bc95'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:19.576254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 923/1837] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:19 2022] DELETE /identity/v3/users/9f0953c56ad94d848e9963c44200bc95 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:19.860630 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:19.862133 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:19.876900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c7d30-3295-4b17-aa49-6fe2878087a0 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:19.937973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:19.948485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505255c-7841-4790-8f91-a178b0271721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:19.949754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505255c-7841-4790-8f91-a178b0271721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:19.950250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505255c-7841-4790-8f91-a178b0271721 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:19.950713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505255c-7841-4790-8f91-a178b0271721 admin admin] PATH_INFO: `/v3/projects/69f59db0b6ed463f8a814400e287bac9` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:19.952219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505255c-7841-4790-8f91-a178b0271721 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69f59db0b6ed463f8a814400e287bac9)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:19.954292 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505255c-7841-4790-8f91-a178b0271721 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:20.002554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2505255c-7841-4790-8f91-a178b0271721 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69f59db0b6ed463f8a814400e287bac9', 'request_id': 'req-2505255c-7841-4790-8f91-a178b0271721'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:20.009172 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2505255c-7841-4790-8f91-a178b0271721 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69f59db0b6ed463f8a814400e287bac9', 'request_id': 'req-2505255c-7841-4790-8f91-a178b0271721'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:20.016967 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2505255c-7841-4790-8f91-a178b0271721 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69f59db0b6ed463f8a814400e287bac9', 'request_id': 'req-2505255c-7841-4790-8f91-a178b0271721'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:20.029754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2505255c-7841-4790-8f91-a178b0271721 admin admin] The token cache is being invalidate because project 69f59db0b6ed463f8a814400e287bac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:20.030065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2505255c-7841-4790-8f91-a178b0271721 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:20.030968 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2505255c-7841-4790-8f91-a178b0271721 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:20.032534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 915/1838] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:19 2022] DELETE /identity/v3/projects/69f59db0b6ed463f8a814400e287bac9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:20.037855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c8cd3-fd74-4adb-9820-55866b75acba None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:20.038227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c8cd3-fd74-4adb-9820-55866b75acba None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:20.038589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c8cd3-fd74-4adb-9820-55866b75acba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:20.057765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-fb2c8cd3-fd74-4adb-9820-55866b75acba None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:20.059192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-fb2c8cd3-fd74-4adb-9820-55866b75acba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:20.126516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 924/1839] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:20.135045 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548495e5-27cc-4e34-8a8f-19ff6fac5165 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:20.135348 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548495e5-27cc-4e34-8a8f-19ff6fac5165 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:20.135639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548495e5-27cc-4e34-8a8f-19ff6fac5165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:20.158221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-548495e5-27cc-4e34-8a8f-19ff6fac5165 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:20.160047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-548495e5-27cc-4e34-8a8f-19ff6fac5165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:20.172723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 916/1840] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:20.173844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c8cd3-fd74-4adb-9820-55866b75acba None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:20.188906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c8cd3-fd74-4adb-9820-55866b75acba None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:20.195391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76356cb9-eec5-4baf-9ef2-a033e5700931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:20.196324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76356cb9-eec5-4baf-9ef2-a033e5700931 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:20.196616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76356cb9-eec5-4baf-9ef2-a033e5700931 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:20.196957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76356cb9-eec5-4baf-9ef2-a033e5700931 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:20.197718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76356cb9-eec5-4baf-9ef2-a033e5700931 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:20.198046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76356cb9-eec5-4baf-9ef2-a033e5700931 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:20.199883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76356cb9-eec5-4baf-9ef2-a033e5700931 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:20.220654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 925/1841] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:55:20.222009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548495e5-27cc-4e34-8a8f-19ff6fac5165 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:20.233588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548495e5-27cc-4e34-8a8f-19ff6fac5165 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:20.242268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ff108-4acb-402c-ba4d-ad7a77337c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:20.243512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ff108-4acb-402c-ba4d-ad7a77337c83 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:20.243797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ff108-4acb-402c-ba4d-ad7a77337c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:20.244164 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ff108-4acb-402c-ba4d-ad7a77337c83 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:20.244933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ff108-4acb-402c-ba4d-ad7a77337c83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:20.247185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ff108-4acb-402c-ba4d-ad7a77337c83 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:20.303803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 917/1842] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:20 2022] POST /identity/v3/projects => generated 393 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:55:20.306200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76356cb9-eec5-4baf-9ef2-a033e5700931 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:20.318906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76356cb9-eec5-4baf-9ef2-a033e5700931 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:20.382140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:20.393205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e504403-734d-48e2-b721-dc1d64674738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:20.393926 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e504403-734d-48e2-b721-dc1d64674738 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:20.394234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e504403-734d-48e2-b721-dc1d64674738 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:20.394518 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e504403-734d-48e2-b721-dc1d64674738 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:20.395542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e504403-734d-48e2-b721-dc1d64674738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:20.397236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e504403-734d-48e2-b721-dc1d64674738 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:20.420953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 926/1843] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:20 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:20.425134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ff108-4acb-402c-ba4d-ad7a77337c83 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:20.455507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ff108-4acb-402c-ba4d-ad7a77337c83 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:20.468612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62110586-7999-424b-9613-942f7a751cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:20.469581 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62110586-7999-424b-9613-942f7a751cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:20.469869 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62110586-7999-424b-9613-942f7a751cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:20.470216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62110586-7999-424b-9613-942f7a751cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:20.471143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62110586-7999-424b-9613-942f7a751cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:20.473054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62110586-7999-424b-9613-942f7a751cc4 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:20.502598 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 918/1844] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:20.505442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e504403-734d-48e2-b721-dc1d64674738 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:20.518599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e504403-734d-48e2-b721-dc1d64674738 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:20.526607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15071db6-beb5-45d1-8030-5212c8517ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:20.527513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15071db6-beb5-45d1-8030-5212c8517ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:20.527846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15071db6-beb5-45d1-8030-5212c8517ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:20.528354 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15071db6-beb5-45d1-8030-5212c8517ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:20.533759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15071db6-beb5-45d1-8030-5212c8517ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:20.535865 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15071db6-beb5-45d1-8030-5212c8517ba4 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:20.573709 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 927/1845] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:20.577331 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62110586-7999-424b-9613-942f7a751cc4 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:20.593393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62110586-7999-424b-9613-942f7a751cc4 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:20.609074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74760b4b-fa9e-4567-b134-ea2b97f31877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:20.609993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74760b4b-fa9e-4567-b134-ea2b97f31877 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:20.610279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74760b4b-fa9e-4567-b134-ea2b97f31877 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:20.610564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74760b4b-fa9e-4567-b134-ea2b97f31877 admin admin] PATH_INFO: `/v3/projects/729a4a05730f42db84c6f437517fb665/users/9b136c2d550e4ab6871af03b16e3bd84/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:20.646426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74760b4b-fa9e-4567-b134-ea2b97f31877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=729a4a05730f42db84c6f437517fb665, user_id=9b136c2d550e4ab6871af03b16e3bd84, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:20.650899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74760b4b-fa9e-4567-b134-ea2b97f31877 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:20.678409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 919/1846] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:20 2022] PUT /identity/v3/projects/729a4a05730f42db84c6f437517fb665/users/9b136c2d550e4ab6871af03b16e3bd84/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:20.690486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c94fb9-0ffd-42ae-97e3-ce5166106aac None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:20.690854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c94fb9-0ffd-42ae-97e3-ce5166106aac None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:20.691176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c94fb9-0ffd-42ae-97e3-ce5166106aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:20.720660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-07c94fb9-0ffd-42ae-97e3-ce5166106aac None None] MFA Rules not processed for user `9b136c2d550e4ab6871af03b16e3bd84`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:20.721853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-07c94fb9-0ffd-42ae-97e3-ce5166106aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:20.798897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 928/1847] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:20.803184 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74760b4b-fa9e-4567-b134-ea2b97f31877 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:20.872479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74760b4b-fa9e-4567-b134-ea2b97f31877 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:20.878203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fad93-9317-406c-a183-c45238052df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:20.879152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fad93-9317-406c-a183-c45238052df2 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:20.879606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fad93-9317-406c-a183-c45238052df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:20.879888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fad93-9317-406c-a183-c45238052df2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:20.880581 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fad93-9317-406c-a183-c45238052df2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:20.882463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fad93-9317-406c-a183-c45238052df2 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:20.910178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 920/1848] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:20 2022] POST /identity/v3/projects => generated 391 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:20.917902 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c94fb9-0ffd-42ae-97e3-ce5166106aac None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:20.935574 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fad93-9317-406c-a183-c45238052df2 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:20.937671 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-223fad93-9317-406c-a183-c45238052df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:20.944708 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:20.955899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c94fb9-0ffd-42ae-97e3-ce5166106aac None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:21.030532 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fad93-9317-406c-a183-c45238052df2 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:21.036614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7d784-1964-4771-b962-0d46b8c8c2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:21.037536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7d784-1964-4771-b962-0d46b8c8c2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:21.037916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415519c-f789-4b40-a208-a6501286d7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:21.038037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7d784-1964-4771-b962-0d46b8c8c2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:21.038213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7d784-1964-4771-b962-0d46b8c8c2d0 admin admin] PATH_INFO: `/v3/users/20f19aa4d350436a98c0bf36d1973143` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:21.038794 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415519c-f789-4b40-a208-a6501286d7ba admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:21.039146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415519c-f789-4b40-a208-a6501286d7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:21.039467 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7d784-1964-4771-b962-0d46b8c8c2d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f19aa4d350436a98c0bf36d1973143)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:21.039563 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415519c-f789-4b40-a208-a6501286d7ba admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:21.040252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415519c-f789-4b40-a208-a6501286d7ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:21.041376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7d784-1964-4771-b962-0d46b8c8c2d0 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:21.042374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415519c-f789-4b40-a208-a6501286d7ba admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:21.070915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 929/1849] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:20 2022] POST /identity/v3/users => generated 302 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:21.076461 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415519c-f789-4b40-a208-a6501286d7ba admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:21.087980 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415519c-f789-4b40-a208-a6501286d7ba admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:21.094690 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd614c93-ef75-45a3-84eb-3f344db85e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:21.095450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614c93-ef75-45a3-84eb-3f344db85e0e admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:21.095736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614c93-ef75-45a3-84eb-3f344db85e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:21.096021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614c93-ef75-45a3-84eb-3f344db85e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:21.096791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd614c93-ef75-45a3-84eb-3f344db85e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:21.098405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd614c93-ef75-45a3-84eb-3f344db85e0e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:21.102149 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-15d7d784-1964-4771-b962-0d46b8c8c2d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f19aa4d350436a98c0bf36d1973143'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:21.114133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 930/1850] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:21.114956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-15d7d784-1964-4771-b962-0d46b8c8c2d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f19aa4d350436a98c0bf36d1973143'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:21.115215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd614c93-ef75-45a3-84eb-3f344db85e0e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:21.116054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-15d7d784-1964-4771-b962-0d46b8c8c2d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f19aa4d350436a98c0bf36d1973143'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:21.125400 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-15d7d784-1964-4771-b962-0d46b8c8c2d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f19aa4d350436a98c0bf36d1973143'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:21.130346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd614c93-ef75-45a3-84eb-3f344db85e0e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:21.135964 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541356b-d0e6-4583-8e26-fe2ab996b6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:21.136977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541356b-d0e6-4583-8e26-fe2ab996b6af admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:21.137358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541356b-d0e6-4583-8e26-fe2ab996b6af admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:21.137646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541356b-d0e6-4583-8e26-fe2ab996b6af admin admin] PATH_INFO: `/v3/projects/9a93f2549ed8432ea7e120e32fb2084c/users/8e37e23dbe02420281ea3033c9e6e50a/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:21.142808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-15d7d784-1964-4771-b962-0d46b8c8c2d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f19aa4d350436a98c0bf36d1973143'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:21.153401 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 921/1851] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:20 2022] DELETE /identity/v3/users/20f19aa4d350436a98c0bf36d1973143 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:21.156333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398377dc-ee13-4757-a659-dd2aad86ce16 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:21.157516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398377dc-ee13-4757-a659-dd2aad86ce16 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:21.158675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398377dc-ee13-4757-a659-dd2aad86ce16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:21.160362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541356b-d0e6-4583-8e26-fe2ab996b6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a93f2549ed8432ea7e120e32fb2084c, user_id=8e37e23dbe02420281ea3033c9e6e50a, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:21.163672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541356b-d0e6-4583-8e26-fe2ab996b6af admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:21.181550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-398377dc-ee13-4757-a659-dd2aad86ce16 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:21.182894 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-398377dc-ee13-4757-a659-dd2aad86ce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:21.190430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 931/1852] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:21 2022] PUT /identity/v3/projects/9a93f2549ed8432ea7e120e32fb2084c/users/8e37e23dbe02420281ea3033c9e6e50a/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:21.193379 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541356b-d0e6-4583-8e26-fe2ab996b6af admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:21.195845 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-6541356b-d0e6-4583-8e26-fe2ab996b6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:21.217303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:21.258184 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 922/1853] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:21.269583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398377dc-ee13-4757-a659-dd2aad86ce16 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:21.271246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-398377dc-ee13-4757-a659-dd2aad86ce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:21.291219 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541356b-d0e6-4583-8e26-fe2ab996b6af admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:21.297246 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12366d-9cbf-4af2-be4e-d4a1b0e1d313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:21.297994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12366d-9cbf-4af2-be4e-d4a1b0e1d313 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:21.298888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12366d-9cbf-4af2-be4e-d4a1b0e1d313 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:21.298888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12366d-9cbf-4af2-be4e-d4a1b0e1d313 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:21.299789 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12366d-9cbf-4af2-be4e-d4a1b0e1d313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:21.301487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12366d-9cbf-4af2-be4e-d4a1b0e1d313 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:21.312570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:21.317442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 932/1854] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:21.318346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12366d-9cbf-4af2-be4e-d4a1b0e1d313 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:21.330505 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12366d-9cbf-4af2-be4e-d4a1b0e1d313 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:21.337011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ed3e5-ecea-411b-8df2-8a3a6be029b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:21.338303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ed3e5-ecea-411b-8df2-8a3a6be029b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:21.338837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ed3e5-ecea-411b-8df2-8a3a6be029b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:21.339396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ed3e5-ecea-411b-8df2-8a3a6be029b6 admin admin] PATH_INFO: `/v3/projects/9a93f2549ed8432ea7e120e32fb2084c/users/8e37e23dbe02420281ea3033c9e6e50a/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:21.342916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ed3e5-ecea-411b-8df2-8a3a6be029b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a93f2549ed8432ea7e120e32fb2084c, user_id=8e37e23dbe02420281ea3033c9e6e50a, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:21.346355 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ed3e5-ecea-411b-8df2-8a3a6be029b6 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:21.353567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398377dc-ee13-4757-a659-dd2aad86ce16 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:21.360280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 933/1855] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:21 2022] PUT /identity/v3/projects/9a93f2549ed8432ea7e120e32fb2084c/users/8e37e23dbe02420281ea3033c9e6e50a/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:21.370074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802b23f-c3c0-4087-9458-bff631902566 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:21.370439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802b23f-c3c0-4087-9458-bff631902566 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:21.370805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802b23f-c3c0-4087-9458-bff631902566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:21.405791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-2802b23f-c3c0-4087-9458-bff631902566 None None] MFA Rules not processed for user `8e37e23dbe02420281ea3033c9e6e50a`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:21.407154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2802b23f-c3c0-4087-9458-bff631902566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:21.426744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38caa5-c718-4046-8cac-3f74cc466e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:21.427742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38caa5-c718-4046-8cac-3f74cc466e8e service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:21.428288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38caa5-c718-4046-8cac-3f74cc466e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:21.428761 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38caa5-c718-4046-8cac-3f74cc466e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:21.503044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 934/1856] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:21 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:21.514347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802b23f-c3c0-4087-9458-bff631902566 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:21.517611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2802b23f-c3c0-4087-9458-bff631902566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:21.586008 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38caa5-c718-4046-8cac-3f74cc466e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:21.588656 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38caa5-c718-4046-8cac-3f74cc466e8e service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:21.601359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 923/1857] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:55:21.625937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802b23f-c3c0-4087-9458-bff631902566 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:21.632394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d202014-a9ae-40d1-b167-f415c4f1d11b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:21.633647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d202014-a9ae-40d1-b167-f415c4f1d11b service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:21.634277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d202014-a9ae-40d1-b167-f415c4f1d11b service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:21.634873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d202014-a9ae-40d1-b167-f415c4f1d11b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:21.651911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d202014-a9ae-40d1-b167-f415c4f1d11b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:21.655338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d202014-a9ae-40d1-b167-f415c4f1d11b service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:21.674254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 935/1858] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:55:21.745900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38caa5-c718-4046-8cac-3f74cc466e8e service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:21.757103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38caa5-c718-4046-8cac-3f74cc466e8e service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:21.761347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a32feb-bd6a-4611-ae4e-dff34abbed9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:21.763273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a32feb-bd6a-4611-ae4e-dff34abbed9a service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:21.763390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a32feb-bd6a-4611-ae4e-dff34abbed9a service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:21.764196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a32feb-bd6a-4611-ae4e-dff34abbed9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:21.766235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e4a32feb-bd6a-4611-ae4e-dff34abbed9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:21.867892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a32feb-bd6a-4611-ae4e-dff34abbed9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:21.869634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a32feb-bd6a-4611-ae4e-dff34abbed9a service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:21.884961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 924/1859] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:22.182470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d202014-a9ae-40d1-b167-f415c4f1d11b service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:22.184011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1d202014-a9ae-40d1-b167-f415c4f1d11b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:22.218182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d202014-a9ae-40d1-b167-f415c4f1d11b service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:22.237500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:22.237731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:22.239185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:22.239185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] PATH_INFO: `/v3/projects/c58c574bbe6042f9be50dd29194adfa3` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:22.245490 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c58c574bbe6042f9be50dd29194adfa3)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:22.247473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:22.305018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:22.356404 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c58c574bbe6042f9be50dd29194adfa3', 'request_id': 'req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:22.368570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c58c574bbe6042f9be50dd29194adfa3', 'request_id': 'req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:22.378646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c58c574bbe6042f9be50dd29194adfa3', 'request_id': 'req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:22.394639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] The token cache is being invalidate because project c58c574bbe6042f9be50dd29194adfa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:22.395043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:22.396132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:22.398478 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 936/1860] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:22 2022] DELETE /identity/v3/projects/c58c574bbe6042f9be50dd29194adfa3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:22.402538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a32feb-bd6a-4611-ae4e-dff34abbed9a service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:22.422224 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a32feb-bd6a-4611-ae4e-dff34abbed9a service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:22.489986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:22.536080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861bf5a-356d-47cc-804a-c10ea93bc74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:22.537026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861bf5a-356d-47cc-804a-c10ea93bc74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:22.537487 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861bf5a-356d-47cc-804a-c10ea93bc74d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:22.537769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861bf5a-356d-47cc-804a-c10ea93bc74d admin admin] PATH_INFO: `/v3/users/3123d03a562048589fdc165c2d6a3910` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:22.539366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c861bf5a-356d-47cc-804a-c10ea93bc74d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3123d03a562048589fdc165c2d6a3910)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:22.541682 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c861bf5a-356d-47cc-804a-c10ea93bc74d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:22.625643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-c861bf5a-356d-47cc-804a-c10ea93bc74d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3123d03a562048589fdc165c2d6a3910'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:22.642270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-c861bf5a-356d-47cc-804a-c10ea93bc74d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3123d03a562048589fdc165c2d6a3910'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:22.643121 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-c861bf5a-356d-47cc-804a-c10ea93bc74d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3123d03a562048589fdc165c2d6a3910'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:22.653942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-c861bf5a-356d-47cc-804a-c10ea93bc74d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3123d03a562048589fdc165c2d6a3910'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:22.676905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-c861bf5a-356d-47cc-804a-c10ea93bc74d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3123d03a562048589fdc165c2d6a3910'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:22.688273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 925/1861] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:22 2022] DELETE /identity/v3/users/3123d03a562048589fdc165c2d6a3910 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:22.890742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:22.893886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:22.917027 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fee8c5-982c-4fbd-8dc1-2584c48cf36e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:22.984178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:22.984966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:22.985277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:22.985518 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] PATH_INFO: `/v3/users/33bc2adb0ce64a27866e1369a8a31a23` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:22.986752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=33bc2adb0ce64a27866e1369a8a31a23)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:22.988771 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:23.038960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33bc2adb0ce64a27866e1369a8a31a23'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:23.047223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33bc2adb0ce64a27866e1369a8a31a23'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:23.054312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33bc2adb0ce64a27866e1369a8a31a23'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:23.055006 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33bc2adb0ce64a27866e1369a8a31a23'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:23.055628 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33bc2adb0ce64a27866e1369a8a31a23'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:23.080276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 937/1862] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:22 2022] DELETE /identity/v3/users/33bc2adb0ce64a27866e1369a8a31a23 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:23.084857 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73863f0-3ae8-49e8-90ac-781830e2f800 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:23.085305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73863f0-3ae8-49e8-90ac-781830e2f800 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:23.085380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73863f0-3ae8-49e8-90ac-781830e2f800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:23.105365 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-b73863f0-3ae8-49e8-90ac-781830e2f800 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:23.106559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b73863f0-3ae8-49e8-90ac-781830e2f800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:23.175935 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 926/1863] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:23.186255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:23.187966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:23.224143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73863f0-3ae8-49e8-90ac-781830e2f800 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:23.225663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b73863f0-3ae8-49e8-90ac-781830e2f800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:23.243899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:23.247119 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73863f0-3ae8-49e8-90ac-781830e2f800 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:23.255215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53374b30-af82-4479-aaec-609b487a123c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:23.256240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53374b30-af82-4479-aaec-609b487a123c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:23.256617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53374b30-af82-4479-aaec-609b487a123c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:23.256996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53374b30-af82-4479-aaec-609b487a123c admin admin] PATH_INFO: `/v3/projects/9fff41c3212144bc81f8e7ccb6ca30b1` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:23.258485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53374b30-af82-4479-aaec-609b487a123c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fff41c3212144bc81f8e7ccb6ca30b1)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:23.261095 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53374b30-af82-4479-aaec-609b487a123c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:23.281598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb91df-b8ca-49a9-b9dc-f133cec624cc admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:23.287818 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563e224-5a87-45c1-98bf-11cfbc6e4e62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:23.288493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563e224-5a87-45c1-98bf-11cfbc6e4e62 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:23.289434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563e224-5a87-45c1-98bf-11cfbc6e4e62 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:23.289751 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563e224-5a87-45c1-98bf-11cfbc6e4e62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:23.311579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563e224-5a87-45c1-98bf-11cfbc6e4e62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:23.313622 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563e224-5a87-45c1-98bf-11cfbc6e4e62 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:23.329064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 938/1864] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:23.338391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:23.345847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-53374b30-af82-4479-aaec-609b487a123c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fff41c3212144bc81f8e7ccb6ca30b1', 'request_id': 'req-53374b30-af82-4479-aaec-609b487a123c'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:23.355364 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-53374b30-af82-4479-aaec-609b487a123c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fff41c3212144bc81f8e7ccb6ca30b1', 'request_id': 'req-53374b30-af82-4479-aaec-609b487a123c'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:23.370292 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-53374b30-af82-4479-aaec-609b487a123c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fff41c3212144bc81f8e7ccb6ca30b1', 'request_id': 'req-53374b30-af82-4479-aaec-609b487a123c'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:23.404053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-53374b30-af82-4479-aaec-609b487a123c admin admin] The token cache is being invalidate because project 9fff41c3212144bc81f8e7ccb6ca30b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:23.405592 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-53374b30-af82-4479-aaec-609b487a123c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:23.406692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-53374b30-af82-4479-aaec-609b487a123c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:23.413617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 927/1865] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:23 2022] DELETE /identity/v3/projects/9fff41c3212144bc81f8e7ccb6ca30b1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:23.422239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70b375-2f06-4a40-b22a-7503db15098b None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:23.422840 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70b375-2f06-4a40-b22a-7503db15098b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:23.423957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70b375-2f06-4a40-b22a-7503db15098b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:23.465491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ec70b375-2f06-4a40-b22a-7503db15098b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:23.469539 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ec70b375-2f06-4a40-b22a-7503db15098b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:23.596801 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 939/1866] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:23.609593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8d740-df4d-4e06-ad1f-914c9e22823b None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:23.610120 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8d740-df4d-4e06-ad1f-914c9e22823b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:23.610524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8d740-df4d-4e06-ad1f-914c9e22823b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:23.645724 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-f0e8d740-df4d-4e06-ad1f-914c9e22823b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:23.647286 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f0e8d740-df4d-4e06-ad1f-914c9e22823b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:23.658458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 928/1867] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:23.662461 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70b375-2f06-4a40-b22a-7503db15098b None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:23.688211 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70b375-2f06-4a40-b22a-7503db15098b None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:23.700906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e1ada-2c40-42c9-9dab-43c33b19b568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:23.702313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e1ada-2c40-42c9-9dab-43c33b19b568 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:23.703159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e1ada-2c40-42c9-9dab-43c33b19b568 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:23.703541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e1ada-2c40-42c9-9dab-43c33b19b568 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:23.704401 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e1ada-2c40-42c9-9dab-43c33b19b568 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:23.707184 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e1ada-2c40-42c9-9dab-43c33b19b568 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:23.710901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e1ada-2c40-42c9-9dab-43c33b19b568 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:23.728969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 940/1868] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:55:23.733881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8d740-df4d-4e06-ad1f-914c9e22823b None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:23.749991 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8d740-df4d-4e06-ad1f-914c9e22823b None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:23.755583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f744034-ebaf-428a-a6f3-8a7d7298f78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:23.756840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f744034-ebaf-428a-a6f3-8a7d7298f78e admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:23.757356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f744034-ebaf-428a-a6f3-8a7d7298f78e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:23.757831 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f744034-ebaf-428a-a6f3-8a7d7298f78e admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:23.758889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f744034-ebaf-428a-a6f3-8a7d7298f78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:23.760889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f744034-ebaf-428a-a6f3-8a7d7298f78e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:23.797951 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 929/1869] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:23 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:55:23.802687 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e1ada-2c40-42c9-9dab-43c33b19b568 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:23.816852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e1ada-2c40-42c9-9dab-43c33b19b568 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:23.902786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a777f-e04f-4007-891a-92113c327a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:23.903676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a777f-e04f-4007-891a-92113c327a28 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:23.903963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a777f-e04f-4007-891a-92113c327a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:23.904386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a777f-e04f-4007-891a-92113c327a28 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:23.905212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a777f-e04f-4007-891a-92113c327a28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:23.907707 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a777f-e04f-4007-891a-92113c327a28 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:23.948666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f744034-ebaf-428a-a6f3-8a7d7298f78e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:23.949450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 941/1870] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:23 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:23.950369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a777f-e04f-4007-891a-92113c327a28 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:23.951179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9f744034-ebaf-428a-a6f3-8a7d7298f78e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:23.963310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a777f-e04f-4007-891a-92113c327a28 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:23.969472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1b2b8-e61e-472b-8027-eaa6d97ea326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:23.970308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1b2b8-e61e-472b-8027-eaa6d97ea326 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:23.971156 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1b2b8-e61e-472b-8027-eaa6d97ea326 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:23.971498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1b2b8-e61e-472b-8027-eaa6d97ea326 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:23.972535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1b2b8-e61e-472b-8027-eaa6d97ea326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:23.976202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1b2b8-e61e-472b-8027-eaa6d97ea326 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:23.986660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f744034-ebaf-428a-a6f3-8a7d7298f78e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:23.989624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 942/1871] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:23.993691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1b2b8-e61e-472b-8027-eaa6d97ea326 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:24.007469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:24.008569 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:24.009005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:24.009857 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] PATH_INFO: `/v3/projects/c5bc2b82879f47ce8132eb70cc0a440b` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:24.013628 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5bc2b82879f47ce8132eb70cc0a440b)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:24.020777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:24.042560 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1b2b8-e61e-472b-8027-eaa6d97ea326 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:24.057356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a4411-63a1-45b3-9857-717497e4762b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:24.058333 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a4411-63a1-45b3-9857-717497e4762b admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:24.065059 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a4411-63a1-45b3-9857-717497e4762b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:24.065555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a4411-63a1-45b3-9857-717497e4762b admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:24.067874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a4411-63a1-45b3-9857-717497e4762b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:24.074574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a4411-63a1-45b3-9857-717497e4762b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:24.100087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 943/1872] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:24.101390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a4411-63a1-45b3-9857-717497e4762b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:24.117191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a4411-63a1-45b3-9857-717497e4762b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:24.122551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c1b19-dead-4c15-bd29-3dfaebd742e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:24.126276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c1b19-dead-4c15-bd29-3dfaebd742e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:24.126721 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c1b19-dead-4c15-bd29-3dfaebd742e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:24.127161 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c1b19-dead-4c15-bd29-3dfaebd742e8 admin admin] PATH_INFO: `/v3/projects/1e9f5107807447f0b0da7d9ff456ccea/users/576a81461dc54dd385743dda946526e6/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:24.149601 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c1b19-dead-4c15-bd29-3dfaebd742e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9f5107807447f0b0da7d9ff456ccea, user_id=576a81461dc54dd385743dda946526e6, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:24.153669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c1b19-dead-4c15-bd29-3dfaebd742e8 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:24.175608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 944/1873] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:24 2022] PUT /identity/v3/projects/1e9f5107807447f0b0da7d9ff456ccea/users/576a81461dc54dd385743dda946526e6/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:24.187433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a85d1e-a2b5-4ba6-91ba-cdd75a2bcd2f None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:24.188210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a85d1e-a2b5-4ba6-91ba-cdd75a2bcd2f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:24.188496 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a85d1e-a2b5-4ba6-91ba-cdd75a2bcd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:24.193818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5bc2b82879f47ce8132eb70cc0a440b', 'request_id': 'req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:24.196383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:24.208678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5bc2b82879f47ce8132eb70cc0a440b', 'request_id': 'req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:24.221604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5bc2b82879f47ce8132eb70cc0a440b', 'request_id': 'req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:24.229899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-b8a85d1e-a2b5-4ba6-91ba-cdd75a2bcd2f None None] MFA Rules not processed for user `576a81461dc54dd385743dda946526e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:24.231577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-b8a85d1e-a2b5-4ba6-91ba-cdd75a2bcd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:24.239317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] The token cache is being invalidate because project c5bc2b82879f47ce8132eb70cc0a440b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:24.239802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:24.240528 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:24.242538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 930/1874] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:23 2022] DELETE /identity/v3/projects/c5bc2b82879f47ce8132eb70cc0a440b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:24.246236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:24.248793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:24.334287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f078c84-6d5d-419f-b6d3-a3e35d34ff50 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:24.351288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 945/1875] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:24.356533 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a85d1e-a2b5-4ba6-91ba-cdd75a2bcd2f None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:24.358319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-b8a85d1e-a2b5-4ba6-91ba-cdd75a2bcd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:24.416236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:24.465865 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a85d1e-a2b5-4ba6-91ba-cdd75a2bcd2f None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:24.471689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d012c-e302-4d7c-9226-ba5ee4f18a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:24.472488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d012c-e302-4d7c-9226-ba5ee4f18a6b admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:24.472959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d012c-e302-4d7c-9226-ba5ee4f18a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:24.473423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d012c-e302-4d7c-9226-ba5ee4f18a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:24.474348 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d012c-e302-4d7c-9226-ba5ee4f18a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:24.476567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d012c-e302-4d7c-9226-ba5ee4f18a6b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:24.519461 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:24.520113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 946/1876] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:24 2022] POST /identity/v3/projects => generated 371 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:24.520597 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:24.520952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:24.521299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] PATH_INFO: `/v3/users/5a4586166f0d42deabc9b3e42d4f3356` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:24.523381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a4586166f0d42deabc9b3e42d4f3356)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:24.526576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:24.527337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d012c-e302-4d7c-9226-ba5ee4f18a6b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:24.608189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a4586166f0d42deabc9b3e42d4f3356'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:24.621083 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a4586166f0d42deabc9b3e42d4f3356'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:24.621867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a4586166f0d42deabc9b3e42d4f3356'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:24.641378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d012c-e302-4d7c-9226-ba5ee4f18a6b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:24.643722 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a4586166f0d42deabc9b3e42d4f3356'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:24.648658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd04180-61bc-443f-aec2-7dd60a4a409b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:24.649641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd04180-61bc-443f-aec2-7dd60a4a409b admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:24.650071 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd04180-61bc-443f-aec2-7dd60a4a409b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:24.650499 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd04180-61bc-443f-aec2-7dd60a4a409b admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:24.651354 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd04180-61bc-443f-aec2-7dd60a4a409b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:24.656687 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd04180-61bc-443f-aec2-7dd60a4a409b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:24.662193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a4586166f0d42deabc9b3e42d4f3356'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:24.672050 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 931/1877] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:24 2022] DELETE /identity/v3/users/5a4586166f0d42deabc9b3e42d4f3356 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:24.715023 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 947/1878] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:24 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:24.727927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:24.729698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:24.830251 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3c1aa-2ebb-4701-966f-3d3826e2f5d0 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:24.835399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73662be2-f491-4ba7-87d9-046e51b4e081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:24.836068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662be2-f491-4ba7-87d9-046e51b4e081 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:24.836409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662be2-f491-4ba7-87d9-046e51b4e081 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:24.836693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73662be2-f491-4ba7-87d9-046e51b4e081 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:24.837431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73662be2-f491-4ba7-87d9-046e51b4e081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:24.839376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73662be2-f491-4ba7-87d9-046e51b4e081 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:24.853225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 932/1879] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:55:24.854936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd04180-61bc-443f-aec2-7dd60a4a409b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:24.875424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd04180-61bc-443f-aec2-7dd60a4a409b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:24.882479 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0ebae-f3b1-4c61-ba2b-cdb4ebc4aa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:24.883420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0ebae-f3b1-4c61-ba2b-cdb4ebc4aa80 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:24.884025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0ebae-f3b1-4c61-ba2b-cdb4ebc4aa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:24.884025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0ebae-f3b1-4c61-ba2b-cdb4ebc4aa80 admin admin] PATH_INFO: `/v3/projects/ab85ba15c1d840dd88e454bddd6c7bc0/users/6bd40640dc3c417cacf03a203f91dad3/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:24.886626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:24.903539 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0ebae-f3b1-4c61-ba2b-cdb4ebc4aa80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab85ba15c1d840dd88e454bddd6c7bc0, user_id=6bd40640dc3c417cacf03a203f91dad3, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:24.906948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0ebae-f3b1-4c61-ba2b-cdb4ebc4aa80 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:24.924685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 948/1880] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:24 2022] PUT /identity/v3/projects/ab85ba15c1d840dd88e454bddd6c7bc0/users/6bd40640dc3c417cacf03a203f91dad3/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:24.930646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73662be2-f491-4ba7-87d9-046e51b4e081 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:24.963797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73662be2-f491-4ba7-87d9-046e51b4e081 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:25.025889 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0ebae-f3b1-4c61-ba2b-cdb4ebc4aa80 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:25.027897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-12c0ebae-f3b1-4c61-ba2b-cdb4ebc4aa80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:25.071336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:25.106619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab31ee-5ac3-4eb1-99b9-9be801e88a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:25.108506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab31ee-5ac3-4eb1-99b9-9be801e88a30 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.108811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab31ee-5ac3-4eb1-99b9-9be801e88a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.109094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab31ee-5ac3-4eb1-99b9-9be801e88a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.110512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab31ee-5ac3-4eb1-99b9-9be801e88a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:25.112179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab31ee-5ac3-4eb1-99b9-9be801e88a30 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:25.126932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 933/1881] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:25.127418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0ebae-f3b1-4c61-ba2b-cdb4ebc4aa80 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:25.130404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab31ee-5ac3-4eb1-99b9-9be801e88a30 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:25.134430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:25.135449 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.135871 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.136872 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f admin admin] PATH_INFO: `/v3/projects/1df27c12146e4fbeb2cf644d61010515` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.138550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1df27c12146e4fbeb2cf644d61010515)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:25.140791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:25.144950 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab31ee-5ac3-4eb1-99b9-9be801e88a30 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:25.156416 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fbd5c-8e22-411a-a1f0-74e7270206f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:25.157716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fbd5c-8e22-411a-a1f0-74e7270206f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.158199 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fbd5c-8e22-411a-a1f0-74e7270206f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.158657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fbd5c-8e22-411a-a1f0-74e7270206f7 admin admin] PATH_INFO: `/v3/projects/ab85ba15c1d840dd88e454bddd6c7bc0/users/6bd40640dc3c417cacf03a203f91dad3/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.161789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fbd5c-8e22-411a-a1f0-74e7270206f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab85ba15c1d840dd88e454bddd6c7bc0, user_id=6bd40640dc3c417cacf03a203f91dad3, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:25.165106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fbd5c-8e22-411a-a1f0-74e7270206f7 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:25.184280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 934/1882] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:25 2022] PUT /identity/v3/projects/ab85ba15c1d840dd88e454bddd6c7bc0/users/6bd40640dc3c417cacf03a203f91dad3/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:25.194987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e06247-3c1c-4e23-bec9-0e5435f38c1f None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.195642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e06247-3c1c-4e23-bec9-0e5435f38c1f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.195734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e06247-3c1c-4e23-bec9-0e5435f38c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.246733 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-38e06247-3c1c-4e23-bec9-0e5435f38c1f None None] MFA Rules not processed for user `6bd40640dc3c417cacf03a203f91dad3`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:25.249041 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1df27c12146e4fbeb2cf644d61010515', 'request_id': 'req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:25.249187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-38e06247-3c1c-4e23-bec9-0e5435f38c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:25.259320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1df27c12146e4fbeb2cf644d61010515', 'request_id': 'req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:25.270692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1df27c12146e4fbeb2cf644d61010515', 'request_id': 'req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:25.288156 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f admin admin] The token cache is being invalidate because project 1df27c12146e4fbeb2cf644d61010515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:25.288458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:25.289429 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ef558334-cee4-4f55-98f4-bfc4a5e5b47f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:25.291563 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 949/1883] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:25 2022] DELETE /identity/v3/projects/1df27c12146e4fbeb2cf644d61010515 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:25.298019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8e53f-f6a1-42d8-89f6-e8ddd043c346 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.298585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8e53f-f6a1-42d8-89f6-e8ddd043c346 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.299264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8e53f-f6a1-42d8-89f6-e8ddd043c346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.335497 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-5da8e53f-f6a1-42d8-89f6-e8ddd043c346 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:25.337042 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5da8e53f-f6a1-42d8-89f6-e8ddd043c346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:25.359533 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 935/1884] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:25 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:25.372122 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e06247-3c1c-4e23-bec9-0e5435f38c1f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:25.374451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-38e06247-3c1c-4e23-bec9-0e5435f38c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:25.408551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 950/1885] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:25.416167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377ed16-96dd-42de-82f2-274c7ab36980 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.416469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377ed16-96dd-42de-82f2-274c7ab36980 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.416780 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377ed16-96dd-42de-82f2-274c7ab36980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.436320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f377ed16-96dd-42de-82f2-274c7ab36980 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:25.437512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f377ed16-96dd-42de-82f2-274c7ab36980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:25.443022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e06247-3c1c-4e23-bec9-0e5435f38c1f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:25.449197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 951/1886] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:25.449349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784f2b9-d9bf-4cca-b4c0-5451008ecc08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:25.449349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784f2b9-d9bf-4cca-b4c0-5451008ecc08 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.449349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784f2b9-d9bf-4cca-b4c0-5451008ecc08 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.449349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784f2b9-d9bf-4cca-b4c0-5451008ecc08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.450741 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377ed16-96dd-42de-82f2-274c7ab36980 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:25.459774 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784f2b9-d9bf-4cca-b4c0-5451008ecc08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:25.460610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377ed16-96dd-42de-82f2-274c7ab36980 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:25.461207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784f2b9-d9bf-4cca-b4c0-5451008ecc08 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:25.465659 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606207a-030a-412e-be55-ee99769b8cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:25.466492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606207a-030a-412e-be55-ee99769b8cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.467170 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606207a-030a-412e-be55-ee99769b8cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.467434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606207a-030a-412e-be55-ee99769b8cc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.468136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606207a-030a-412e-be55-ee99769b8cc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:25.468439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606207a-030a-412e-be55-ee99769b8cc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:25.470056 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606207a-030a-412e-be55-ee99769b8cc3 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:25.472732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:25.472868 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 936/1887] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:55:25.503033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 952/1888] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:55:25.507830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784f2b9-d9bf-4cca-b4c0-5451008ecc08 service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:25.519156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784f2b9-d9bf-4cca-b4c0-5451008ecc08 service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:25.525821 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcce1f-7e78-4e01-8c52-3a493a96a461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:25.526485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcce1f-7e78-4e01-8c52-3a493a96a461 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.526771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcce1f-7e78-4e01-8c52-3a493a96a461 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.527112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcce1f-7e78-4e01-8c52-3a493a96a461 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.527802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcce1f-7e78-4e01-8c52-3a493a96a461 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:25.529740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcce1f-7e78-4e01-8c52-3a493a96a461 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:25.568497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 937/1889] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:25 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:25.569468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606207a-030a-412e-be55-ee99769b8cc3 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:25.584231 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606207a-030a-412e-be55-ee99769b8cc3 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:25.694779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a14b5-be6c-4829-be06-ce61813e50c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:25.695506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a14b5-be6c-4829-be06-ce61813e50c6 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.695926 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a14b5-be6c-4829-be06-ce61813e50c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.696207 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a14b5-be6c-4829-be06-ce61813e50c6 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.696944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a14b5-be6c-4829-be06-ce61813e50c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:25.698957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a14b5-be6c-4829-be06-ce61813e50c6 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:25.722904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 953/1890] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:25 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:25.727361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcce1f-7e78-4e01-8c52-3a493a96a461 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:25.739319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcce1f-7e78-4e01-8c52-3a493a96a461 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:25.752039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02afb3-0eaa-49ec-a70e-91e58099dace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:25.752919 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02afb3-0eaa-49ec-a70e-91e58099dace admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.754336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02afb3-0eaa-49ec-a70e-91e58099dace admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.754916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02afb3-0eaa-49ec-a70e-91e58099dace admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.756838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02afb3-0eaa-49ec-a70e-91e58099dace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:25.758823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02afb3-0eaa-49ec-a70e-91e58099dace admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:25.777307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 938/1891] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:25.778894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a14b5-be6c-4829-be06-ce61813e50c6 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:25.793374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a14b5-be6c-4829-be06-ce61813e50c6 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:25.800122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557454a0-581b-41cd-a49a-cbf7084a9b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:25.801713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557454a0-581b-41cd-a49a-cbf7084a9b12 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.802106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557454a0-581b-41cd-a49a-cbf7084a9b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.802592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557454a0-581b-41cd-a49a-cbf7084a9b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.803621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557454a0-581b-41cd-a49a-cbf7084a9b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:25.807487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557454a0-581b-41cd-a49a-cbf7084a9b12 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:25.823428 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 954/1892] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:25.828696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02afb3-0eaa-49ec-a70e-91e58099dace admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:25.841242 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02afb3-0eaa-49ec-a70e-91e58099dace admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:25.848459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaad5ea-70de-4661-890d-99370266cdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:25.849363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaad5ea-70de-4661-890d-99370266cdbd admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.849653 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaad5ea-70de-4661-890d-99370266cdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.849932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaad5ea-70de-4661-890d-99370266cdbd admin admin] PATH_INFO: `/v3/projects/8494c98a48ed4496b058be30bb20541e/users/5d95fbc2347946398ec5b360b957bf07/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.867021 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaad5ea-70de-4661-890d-99370266cdbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8494c98a48ed4496b058be30bb20541e, user_id=5d95fbc2347946398ec5b360b957bf07, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:25.871627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaad5ea-70de-4661-890d-99370266cdbd admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:25.886555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 939/1893] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:25 2022] PUT /identity/v3/projects/8494c98a48ed4496b058be30bb20541e/users/5d95fbc2347946398ec5b360b957bf07/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:25.898829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ce391-66dc-43e5-81ce-e0cfbe44d703 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:25.899249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ce391-66dc-43e5-81ce-e0cfbe44d703 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:25.899637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ce391-66dc-43e5-81ce-e0cfbe44d703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:25.933830 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-a25ce391-66dc-43e5-81ce-e0cfbe44d703 None None] MFA Rules not processed for user `5d95fbc2347946398ec5b360b957bf07`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:25.935123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a25ce391-66dc-43e5-81ce-e0cfbe44d703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:26.017193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaad5ea-70de-4661-890d-99370266cdbd admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:26.019208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-cfaad5ea-70de-4661-890d-99370266cdbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:26.021114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 955/1894] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:26.024911 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:26.026476 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ce391-66dc-43e5-81ce-e0cfbe44d703 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:26.101109 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaad5ea-70de-4661-890d-99370266cdbd admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:26.106361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:26.107944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ce391-66dc-43e5-81ce-e0cfbe44d703 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:26.108318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:26.108752 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:26.109413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] PATH_INFO: `/v3/users/9b136c2d550e4ab6871af03b16e3bd84` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:26.110484 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b136c2d550e4ab6871af03b16e3bd84)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:26.112799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:26.115418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50eb6f-bdea-4989-8204-f8374336e0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:26.116315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50eb6f-bdea-4989-8204-f8374336e0af admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:26.116616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50eb6f-bdea-4989-8204-f8374336e0af admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:26.116885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50eb6f-bdea-4989-8204-f8374336e0af admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:26.117650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50eb6f-bdea-4989-8204-f8374336e0af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:26.119975 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50eb6f-bdea-4989-8204-f8374336e0af admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:26.125617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:26.159238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 956/1895] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:26 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:26.159238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50eb6f-bdea-4989-8204-f8374336e0af admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:26.159238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4f50eb6f-bdea-4989-8204-f8374336e0af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:26.173662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b136c2d550e4ab6871af03b16e3bd84'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:26.181096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b136c2d550e4ab6871af03b16e3bd84'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:26.181774 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b136c2d550e4ab6871af03b16e3bd84'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:26.191491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b136c2d550e4ab6871af03b16e3bd84'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:26.207352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b136c2d550e4ab6871af03b16e3bd84'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:26.213818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 940/1896] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:26 2022] DELETE /identity/v3/users/9b136c2d550e4ab6871af03b16e3bd84 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:26.214734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:26.216172 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:26.230744 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50eb6f-bdea-4989-8204-f8374336e0af admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:26.300345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:26.301788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:26.302117 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:26.302414 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] PATH_INFO: `/v3/users/576a81461dc54dd385743dda946526e6` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:26.304201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=576a81461dc54dd385743dda946526e6)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:26.306361 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:26.319199 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3fa81-fc32-40c5-96aa-58da0fc57e0a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:26.325760 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e9d82-ff2b-4677-8f2f-c63481a90ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:26.327152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e9d82-ff2b-4677-8f2f-c63481a90ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:26.327439 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e9d82-ff2b-4677-8f2f-c63481a90ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:26.327717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e9d82-ff2b-4677-8f2f-c63481a90ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:26.328414 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e9d82-ff2b-4677-8f2f-c63481a90ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:26.330370 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e9d82-ff2b-4677-8f2f-c63481a90ad2 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:26.360832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 941/1897] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:26 2022] POST /identity/v3/users => generated 294 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:26.360832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5eeef-066c-4ee5-87a6-2b203de2b2fd None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:26.360832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5eeef-066c-4ee5-87a6-2b203de2b2fd None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:26.360832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5eeef-066c-4ee5-87a6-2b203de2b2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:26.379282 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '576a81461dc54dd385743dda946526e6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:26.391024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-04f5eeef-066c-4ee5-87a6-2b203de2b2fd None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:26.391024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-04f5eeef-066c-4ee5-87a6-2b203de2b2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:26.391369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '576a81461dc54dd385743dda946526e6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:26.401707 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '576a81461dc54dd385743dda946526e6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:26.403337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '576a81461dc54dd385743dda946526e6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:26.405058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '576a81461dc54dd385743dda946526e6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:26.411517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 942/1898] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:26.411517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5eeef-066c-4ee5-87a6-2b203de2b2fd None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:26.426949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:26.432959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5eeef-066c-4ee5-87a6-2b203de2b2fd None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:26.439609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 957/1899] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:26 2022] DELETE /identity/v3/users/576a81461dc54dd385743dda946526e6 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:26.440480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:26.440809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca675b-a3ae-4b6f-947a-a46bae74cd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:26.441769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca675b-a3ae-4b6f-947a-a46bae74cd58 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:26.441909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:26.442057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca675b-a3ae-4b6f-947a-a46bae74cd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:26.442427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca675b-a3ae-4b6f-947a-a46bae74cd58 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:26.443509 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca675b-a3ae-4b6f-947a-a46bae74cd58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:26.445495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca675b-a3ae-4b6f-947a-a46bae74cd58 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:26.457113 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 943/1900] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:26.458407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d629ea-5079-4d51-9405-2f5a7291035c None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:26.458696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d629ea-5079-4d51-9405-2f5a7291035c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:26.459319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d629ea-5079-4d51-9405-2f5a7291035c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:26.460821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:26.486688 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-71d629ea-5079-4d51-9405-2f5a7291035c None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:26.488064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-71d629ea-5079-4d51-9405-2f5a7291035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:26.516248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b770eee-4816-4efa-9357-629d177b81ed admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:26.520990 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0cecb-4cd9-4305-9d42-48450f5dc7f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:26.521868 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0cecb-4cd9-4305-9d42-48450f5dc7f8 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:26.522305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0cecb-4cd9-4305-9d42-48450f5dc7f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:26.522724 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0cecb-4cd9-4305-9d42-48450f5dc7f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:26.525171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-40d0cecb-4cd9-4305-9d42-48450f5dc7f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:26.561213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 944/1901] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:26.562144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d629ea-5079-4d51-9405-2f5a7291035c None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:26.563453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-71d629ea-5079-4d51-9405-2f5a7291035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:26.576376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d629ea-5079-4d51-9405-2f5a7291035c None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:26.582619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b6aca-5268-4364-8ccc-8f3b268ae1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:26.583405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b6aca-5268-4364-8ccc-8f3b268ae1ec admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:26.583692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b6aca-5268-4364-8ccc-8f3b268ae1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:26.583974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b6aca-5268-4364-8ccc-8f3b268ae1ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:26.584713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b6aca-5268-4364-8ccc-8f3b268ae1ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:26.586306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b6aca-5268-4364-8ccc-8f3b268ae1ec admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:26.597761 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 945/1902] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:26.598601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b6aca-5268-4364-8ccc-8f3b268ae1ec admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:26.602335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-af7b6aca-5268-4364-8ccc-8f3b268ae1ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:26.602847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0cecb-4cd9-4305-9d42-48450f5dc7f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:26.604475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0cecb-4cd9-4305-9d42-48450f5dc7f8 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:26.614825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b6aca-5268-4364-8ccc-8f3b268ae1ec admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:26.617133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 958/1903] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:26.617834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0cecb-4cd9-4305-9d42-48450f5dc7f8 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:26.619222 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3a44a-9152-44bb-822f-4af6bd71cbe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:26.619871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3a44a-9152-44bb-822f-4af6bd71cbe4 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:26.620170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3a44a-9152-44bb-822f-4af6bd71cbe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:26.620421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3a44a-9152-44bb-822f-4af6bd71cbe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:26.629858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0cecb-4cd9-4305-9d42-48450f5dc7f8 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:26.636888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f641fd1-6cea-4ee2-a7c3-4436c1c7eceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:26.637837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f641fd1-6cea-4ee2-a7c3-4436c1c7eceb admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:26.638128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f641fd1-6cea-4ee2-a7c3-4436c1c7eceb admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:26.638549 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f641fd1-6cea-4ee2-a7c3-4436c1c7eceb admin admin] PATH_INFO: `/v3/projects/c8765c5c581645a181effaadc2f17354/users/cffeb4e38b774419be10efe71b54c147/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:26.648677 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3a44a-9152-44bb-822f-4af6bd71cbe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:26.651806 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3a44a-9152-44bb-822f-4af6bd71cbe4 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:26.656087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f641fd1-6cea-4ee2-a7c3-4436c1c7eceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8765c5c581645a181effaadc2f17354, user_id=cffeb4e38b774419be10efe71b54c147, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:26.659486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f641fd1-6cea-4ee2-a7c3-4436c1c7eceb admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:26.686571 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 959/1904] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:26 2022] PUT /identity/v3/projects/c8765c5c581645a181effaadc2f17354/users/cffeb4e38b774419be10efe71b54c147/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:26.689446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf1f18-23fb-48c7-8754-534c262ac4ec None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:26.689834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf1f18-23fb-48c7-8754-534c262ac4ec None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:26.690257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf1f18-23fb-48c7-8754-534c262ac4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:26.702050 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 946/1905] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:26.730127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-0fcf1f18-23fb-48c7-8754-534c262ac4ec None None] MFA Rules not processed for user `cffeb4e38b774419be10efe71b54c147`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:26.731758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-0fcf1f18-23fb-48c7-8754-534c262ac4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:26.817132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 960/1906] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:26.818626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3a44a-9152-44bb-822f-4af6bd71cbe4 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:26.832357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3a44a-9152-44bb-822f-4af6bd71cbe4 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:26.923625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f52a74-efee-4994-9010-f92f2df60503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:26.924613 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f52a74-efee-4994-9010-f92f2df60503 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:26.924997 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f52a74-efee-4994-9010-f92f2df60503 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:26.925413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f52a74-efee-4994-9010-f92f2df60503 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:26.927352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f52a74-efee-4994-9010-f92f2df60503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:26.930150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f52a74-efee-4994-9010-f92f2df60503 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:26.984418 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 947/1907] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:26 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:26.988330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf1f18-23fb-48c7-8754-534c262ac4ec None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:27.048959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf1f18-23fb-48c7-8754-534c262ac4ec None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:27.095062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:27.149814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3d88b-ef4c-4b0e-adf2-4fd875e109fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:27.150918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d88b-ef4c-4b0e-adf2-4fd875e109fc admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:27.151119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d88b-ef4c-4b0e-adf2-4fd875e109fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:27.151492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d88b-ef4c-4b0e-adf2-4fd875e109fc admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:27.152336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3d88b-ef4c-4b0e-adf2-4fd875e109fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:27.155012 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3d88b-ef4c-4b0e-adf2-4fd875e109fc admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:27.192364 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 961/1908] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:26 2022] POST /identity/v3/users => generated 299 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:27.196565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f52a74-efee-4994-9010-f92f2df60503 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:27.215194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f52a74-efee-4994-9010-f92f2df60503 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:27.235210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917f573-6c52-4070-90ea-464290627965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:27.236163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917f573-6c52-4070-90ea-464290627965 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:27.236657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917f573-6c52-4070-90ea-464290627965 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:27.237115 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917f573-6c52-4070-90ea-464290627965 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:27.242194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917f573-6c52-4070-90ea-464290627965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:27.244079 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917f573-6c52-4070-90ea-464290627965 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:27.261887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 948/1909] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:27.267336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3d88b-ef4c-4b0e-adf2-4fd875e109fc admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:27.276639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917f573-6c52-4070-90ea-464290627965 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:27.280454 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-0917f573-6c52-4070-90ea-464290627965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:27.285531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3d88b-ef4c-4b0e-adf2-4fd875e109fc admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:27.292158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64980538-b719-460d-9806-cf849d71c630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:27.308032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64980538-b719-460d-9806-cf849d71c630 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:27.308402 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64980538-b719-460d-9806-cf849d71c630 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:27.309019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64980538-b719-460d-9806-cf849d71c630 admin admin] PATH_INFO: `/v3/projects/654a0dfeade24088aff933999c6452f7/users/a0f5fefbafb44577b5159f7fed92a3ff/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:27.328077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917f573-6c52-4070-90ea-464290627965 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:27.340333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:27.349526 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:27.350098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:27.350227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64980538-b719-460d-9806-cf849d71c630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=654a0dfeade24088aff933999c6452f7, user_id=a0f5fefbafb44577b5159f7fed92a3ff, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:27.350987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] PATH_INFO: `/v3/projects/729a4a05730f42db84c6f437517fb665` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:27.352983 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64980538-b719-460d-9806-cf849d71c630 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:27.353298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] RBAC: Authorizing `identity:delete_project(project_id=729a4a05730f42db84c6f437517fb665)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:27.355696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:27.383317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 962/1910] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:27 2022] PUT /identity/v3/projects/654a0dfeade24088aff933999c6452f7/users/a0f5fefbafb44577b5159f7fed92a3ff/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:27.400494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64980538-b719-460d-9806-cf849d71c630 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:27.459791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:27.474377 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '729a4a05730f42db84c6f437517fb665', 'request_id': 'req-48c3e391-4316-445e-bb2b-1e871f7fa88b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:27.489846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '729a4a05730f42db84c6f437517fb665', 'request_id': 'req-48c3e391-4316-445e-bb2b-1e871f7fa88b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:27.505390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '729a4a05730f42db84c6f437517fb665', 'request_id': 'req-48c3e391-4316-445e-bb2b-1e871f7fa88b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:27.527337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] The token cache is being invalidate because project 729a4a05730f42db84c6f437517fb665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:27.528221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:27.530207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:27.531079 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64980538-b719-460d-9806-cf849d71c630 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:27.533581 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 949/1911] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:27 2022] DELETE /identity/v3/projects/729a4a05730f42db84c6f437517fb665 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:27.537207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:27.539973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:27.614694 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26803ab2-3abd-49d2-a6ae-c002ca589c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:27.617405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26803ab2-3abd-49d2-a6ae-c002ca589c3d admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:27.617855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26803ab2-3abd-49d2-a6ae-c002ca589c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:27.620359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26803ab2-3abd-49d2-a6ae-c002ca589c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:27.621332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26803ab2-3abd-49d2-a6ae-c002ca589c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:27.623196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26803ab2-3abd-49d2-a6ae-c002ca589c3d admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:27.635211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3e391-4316-445e-bb2b-1e871f7fa88b admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:27.638011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 963/1912] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:27.645487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26803ab2-3abd-49d2-a6ae-c002ca589c3d admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:27.653538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847e700-1c7e-462d-aacf-88b005c9ca18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:27.655475 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847e700-1c7e-462d-aacf-88b005c9ca18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:27.656098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-26803ab2-3abd-49d2-a6ae-c002ca589c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:27.656480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847e700-1c7e-462d-aacf-88b005c9ca18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:27.657106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847e700-1c7e-462d-aacf-88b005c9ca18 admin admin] PATH_INFO: `/v3/users/8e37e23dbe02420281ea3033c9e6e50a` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:27.658769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847e700-1c7e-462d-aacf-88b005c9ca18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e37e23dbe02420281ea3033c9e6e50a)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:27.661652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847e700-1c7e-462d-aacf-88b005c9ca18 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:27.679219 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26803ab2-3abd-49d2-a6ae-c002ca589c3d admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:27.686828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98aff61-4c01-411d-9cb2-ef97f58bcccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:27.687829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aff61-4c01-411d-9cb2-ef97f58bcccb admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:27.688332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aff61-4c01-411d-9cb2-ef97f58bcccb admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:27.688836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aff61-4c01-411d-9cb2-ef97f58bcccb admin admin] PATH_INFO: `/v3/projects/654a0dfeade24088aff933999c6452f7/users/a0f5fefbafb44577b5159f7fed92a3ff/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:27.695062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98aff61-4c01-411d-9cb2-ef97f58bcccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=654a0dfeade24088aff933999c6452f7, user_id=a0f5fefbafb44577b5159f7fed92a3ff, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:27.697130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98aff61-4c01-411d-9cb2-ef97f58bcccb admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:27.733201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 964/1913] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:27 2022] PUT /identity/v3/projects/654a0dfeade24088aff933999c6452f7/users/a0f5fefbafb44577b5159f7fed92a3ff/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:27.733201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98aff61-4c01-411d-9cb2-ef97f58bcccb admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:27.742393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f98aff61-4c01-411d-9cb2-ef97f58bcccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:27.794458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4847e700-1c7e-462d-aacf-88b005c9ca18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e37e23dbe02420281ea3033c9e6e50a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:27.802107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:27.805295 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4847e700-1c7e-462d-aacf-88b005c9ca18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e37e23dbe02420281ea3033c9e6e50a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:27.806254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4847e700-1c7e-462d-aacf-88b005c9ca18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e37e23dbe02420281ea3033c9e6e50a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:27.816561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4847e700-1c7e-462d-aacf-88b005c9ca18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e37e23dbe02420281ea3033c9e6e50a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:27.833713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4847e700-1c7e-462d-aacf-88b005c9ca18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e37e23dbe02420281ea3033c9e6e50a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:27.840671 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 950/1914] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:27 2022] DELETE /identity/v3/users/8e37e23dbe02420281ea3033c9e6e50a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:27.842071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67016d4a-8d7a-4e0d-9859-14490e398768 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:27.842519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67016d4a-8d7a-4e0d-9859-14490e398768 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:27.842922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67016d4a-8d7a-4e0d-9859-14490e398768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:27.858301 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98aff61-4c01-411d-9cb2-ef97f58bcccb admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:27.877127 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-67016d4a-8d7a-4e0d-9859-14490e398768 None None] MFA Rules not processed for user `a0f5fefbafb44577b5159f7fed92a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:27.878852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-67016d4a-8d7a-4e0d-9859-14490e398768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:27.924601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:27.942646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:27.943300 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:27.943727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:27.943903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] PATH_INFO: `/v3/projects/1e9f5107807447f0b0da7d9ff456ccea` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:27.949333 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9f5107807447f0b0da7d9ff456ccea)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:27.952473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:27.969920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 951/1915] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:27.988703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67016d4a-8d7a-4e0d-9859-14490e398768 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:27.990620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-67016d4a-8d7a-4e0d-9859-14490e398768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:28.032517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9f5107807447f0b0da7d9ff456ccea', 'request_id': 'req-ced700fc-801b-4dc7-93b8-86cbfa140917'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.038359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9f5107807447f0b0da7d9ff456ccea', 'request_id': 'req-ced700fc-801b-4dc7-93b8-86cbfa140917'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.049525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9f5107807447f0b0da7d9ff456ccea', 'request_id': 'req-ced700fc-801b-4dc7-93b8-86cbfa140917'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.073500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] The token cache is being invalidate because project 1e9f5107807447f0b0da7d9ff456ccea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:28.073792 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.074471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.076383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 965/1916] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:27 2022] DELETE /identity/v3/projects/1e9f5107807447f0b0da7d9ff456ccea => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:28.080254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:28.081926 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:28.099485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67016d4a-8d7a-4e0d-9859-14490e398768 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:28.162358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11bf59-cb05-4c5c-bc5d-d6218e44f802 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:28.163346 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11bf59-cb05-4c5c-bc5d-d6218e44f802 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:28.163796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11bf59-cb05-4c5c-bc5d-d6218e44f802 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:28.164358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11bf59-cb05-4c5c-bc5d-d6218e44f802 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:28.168323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6e11bf59-cb05-4c5c-bc5d-d6218e44f802 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:28.211306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced700fc-801b-4dc7-93b8-86cbfa140917 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:28.221430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0276422-2051-421e-98db-2433bc77929a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:28.222330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0276422-2051-421e-98db-2433bc77929a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:28.222645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0276422-2051-421e-98db-2433bc77929a admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:28.222927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0276422-2051-421e-98db-2433bc77929a admin admin] PATH_INFO: `/v3/users/6bd40640dc3c417cacf03a203f91dad3` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:28.225398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0276422-2051-421e-98db-2433bc77929a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bd40640dc3c417cacf03a203f91dad3)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:28.227874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0276422-2051-421e-98db-2433bc77929a admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:28.231224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:28.262908 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11bf59-cb05-4c5c-bc5d-d6218e44f802 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:28.265669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11bf59-cb05-4c5c-bc5d-d6218e44f802 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:28.276434 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 952/1917] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:55:28.277359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11bf59-cb05-4c5c-bc5d-d6218e44f802 service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:28.278806 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6e11bf59-cb05-4c5c-bc5d-d6218e44f802 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:28.297186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11bf59-cb05-4c5c-bc5d-d6218e44f802 service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:28.303391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d063531-3760-4d49-8ba2-cdb19fbffac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:28.304369 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d063531-3760-4d49-8ba2-cdb19fbffac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:28.305666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d063531-3760-4d49-8ba2-cdb19fbffac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:28.306011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d063531-3760-4d49-8ba2-cdb19fbffac8 admin admin] PATH_INFO: `/v3/projects/9a93f2549ed8432ea7e120e32fb2084c` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:28.307379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d063531-3760-4d49-8ba2-cdb19fbffac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a93f2549ed8432ea7e120e32fb2084c)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:28.309704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d063531-3760-4d49-8ba2-cdb19fbffac8 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:28.365627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f0276422-2051-421e-98db-2433bc77929a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bd40640dc3c417cacf03a203f91dad3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.383046 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:28.383773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f0276422-2051-421e-98db-2433bc77929a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bd40640dc3c417cacf03a203f91dad3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.407439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f0276422-2051-421e-98db-2433bc77929a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bd40640dc3c417cacf03a203f91dad3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.407867 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f0276422-2051-421e-98db-2433bc77929a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bd40640dc3c417cacf03a203f91dad3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.409614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f0276422-2051-421e-98db-2433bc77929a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bd40640dc3c417cacf03a203f91dad3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.420158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3d063531-3760-4d49-8ba2-cdb19fbffac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a93f2549ed8432ea7e120e32fb2084c', 'request_id': 'req-3d063531-3760-4d49-8ba2-cdb19fbffac8'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.446036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3d063531-3760-4d49-8ba2-cdb19fbffac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a93f2549ed8432ea7e120e32fb2084c', 'request_id': 'req-3d063531-3760-4d49-8ba2-cdb19fbffac8'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.450528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 966/1918] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:28 2022] DELETE /identity/v3/users/6bd40640dc3c417cacf03a203f91dad3 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:28.472031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3d063531-3760-4d49-8ba2-cdb19fbffac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a93f2549ed8432ea7e120e32fb2084c', 'request_id': 'req-3d063531-3760-4d49-8ba2-cdb19fbffac8'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.502340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3d063531-3760-4d49-8ba2-cdb19fbffac8 admin admin] The token cache is being invalidate because project 9a93f2549ed8432ea7e120e32fb2084c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:28.503338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3d063531-3760-4d49-8ba2-cdb19fbffac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.506537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3d063531-3760-4d49-8ba2-cdb19fbffac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:28.510499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 953/1919] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:28 2022] DELETE /identity/v3/projects/9a93f2549ed8432ea7e120e32fb2084c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:28.515795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d83d2-758e-4d32-a4ea-9c5b09bf325a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:28.516343 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d83d2-758e-4d32-a4ea-9c5b09bf325a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:28.516716 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d83d2-758e-4d32-a4ea-9c5b09bf325a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:28.547279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-496d83d2-758e-4d32-a4ea-9c5b09bf325a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:28.548928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-496d83d2-758e-4d32-a4ea-9c5b09bf325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:28.653299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 967/1920] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:28.661948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033859e-33fb-4154-a30e-f3d3b9f5ec41 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:28.662934 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033859e-33fb-4154-a30e-f3d3b9f5ec41 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:28.663541 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033859e-33fb-4154-a30e-f3d3b9f5ec41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:28.720860 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-d033859e-33fb-4154-a30e-f3d3b9f5ec41 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:28.722734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d033859e-33fb-4154-a30e-f3d3b9f5ec41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:28.743446 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 954/1921] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:28.748854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d83d2-758e-4d32-a4ea-9c5b09bf325a None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:28.763095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d83d2-758e-4d32-a4ea-9c5b09bf325a None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:28.771705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3551a9-3440-44b9-b3f8-4314b453bb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:28.775453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3551a9-3440-44b9-b3f8-4314b453bb1f admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:28.775853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3551a9-3440-44b9-b3f8-4314b453bb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:28.776270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3551a9-3440-44b9-b3f8-4314b453bb1f admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:28.777810 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3551a9-3440-44b9-b3f8-4314b453bb1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:28.791198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3551a9-3440-44b9-b3f8-4314b453bb1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:28.795727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3551a9-3440-44b9-b3f8-4314b453bb1f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:28.837250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 968/1922] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:55:28.841643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033859e-33fb-4154-a30e-f3d3b9f5ec41 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:28.857724 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033859e-33fb-4154-a30e-f3d3b9f5ec41 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:28.872567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c5360-60be-44b2-a6e4-e2888b54e286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:28.874383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c5360-60be-44b2-a6e4-e2888b54e286 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:28.876292 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c5360-60be-44b2-a6e4-e2888b54e286 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:28.876515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c5360-60be-44b2-a6e4-e2888b54e286 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:28.877334 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c5360-60be-44b2-a6e4-e2888b54e286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:28.883447 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c5360-60be-44b2-a6e4-e2888b54e286 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:28.947643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 955/1923] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:28 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:28.951723 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3551a9-3440-44b9-b3f8-4314b453bb1f admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:28.978653 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3551a9-3440-44b9-b3f8-4314b453bb1f admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:29.144728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b291b-2c7c-4392-bc24-c041191d8d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:29.145591 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b291b-2c7c-4392-bc24-c041191d8d51 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:29.146035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b291b-2c7c-4392-bc24-c041191d8d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:29.146410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b291b-2c7c-4392-bc24-c041191d8d51 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:29.147366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b291b-2c7c-4392-bc24-c041191d8d51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:29.150316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b291b-2c7c-4392-bc24-c041191d8d51 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:29.177024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c5360-60be-44b2-a6e4-e2888b54e286 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:29.189335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-434c5360-60be-44b2-a6e4-e2888b54e286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:29.249515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c5360-60be-44b2-a6e4-e2888b54e286 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:29.251598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:29.259530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:29.260645 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:29.261123 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:29.262011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] PATH_INFO: `/v3/projects/ab85ba15c1d840dd88e454bddd6c7bc0` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:29.264033 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab85ba15c1d840dd88e454bddd6c7bc0)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:29.269494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:29.273988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 969/1924] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:28 2022] POST /identity/v3/users => generated 294 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:29.276604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b291b-2c7c-4392-bc24-c041191d8d51 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:29.301683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b291b-2c7c-4392-bc24-c041191d8d51 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:29.309985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35fcad-040f-48d9-be5e-95bcff983bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:29.313106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35fcad-040f-48d9-be5e-95bcff983bcb admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:29.313106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35fcad-040f-48d9-be5e-95bcff983bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:29.313341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35fcad-040f-48d9-be5e-95bcff983bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:29.314000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35fcad-040f-48d9-be5e-95bcff983bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:29.316897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35fcad-040f-48d9-be5e-95bcff983bcb admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:29.328957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 970/1925] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:29.336677 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35fcad-040f-48d9-be5e-95bcff983bcb admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:29.348565 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35fcad-040f-48d9-be5e-95bcff983bcb admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:29.352929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab85ba15c1d840dd88e454bddd6c7bc0', 'request_id': 'req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:29.364298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab85ba15c1d840dd88e454bddd6c7bc0', 'request_id': 'req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:29.373970 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab85ba15c1d840dd88e454bddd6c7bc0', 'request_id': 'req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:29.393127 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] The token cache is being invalidate because project ab85ba15c1d840dd88e454bddd6c7bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:29.393896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:29.398212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:29.399066 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 956/1926] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:29 2022] DELETE /identity/v3/projects/ab85ba15c1d840dd88e454bddd6c7bc0 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:29.403854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:29.406113 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:29.412759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f0732-69ba-43b8-a45f-4a05761c44e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:29.413648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f0732-69ba-43b8-a45f-4a05761c44e4 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:29.414072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f0732-69ba-43b8-a45f-4a05761c44e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:29.414366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f0732-69ba-43b8-a45f-4a05761c44e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:29.415132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f0732-69ba-43b8-a45f-4a05761c44e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:29.416732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f0732-69ba-43b8-a45f-4a05761c44e4 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:29.427611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 971/1927] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:29.428746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fd690-0f9b-4037-9095-4dc9772ea9e4 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:29.429039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fd690-0f9b-4037-9095-4dc9772ea9e4 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:29.429321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fd690-0f9b-4037-9095-4dc9772ea9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:29.449488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-890fd690-0f9b-4037-9095-4dc9772ea9e4 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:29.451852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-890fd690-0f9b-4037-9095-4dc9772ea9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:29.461455 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 972/1928] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:29.462068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fd690-0f9b-4037-9095-4dc9772ea9e4 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:29.463321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-890fd690-0f9b-4037-9095-4dc9772ea9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:29.471997 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c67dc1-8eb4-4227-969e-c0b4cd39c297 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:29.475733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fd690-0f9b-4037-9095-4dc9772ea9e4 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:29.476080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7da19-89b9-4b89-98ef-13d3ff7e2743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:29.476895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7da19-89b9-4b89-98ef-13d3ff7e2743 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:29.477287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7da19-89b9-4b89-98ef-13d3ff7e2743 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:29.477657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7da19-89b9-4b89-98ef-13d3ff7e2743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:29.479708 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d0f7da19-89b9-4b89-98ef-13d3ff7e2743 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:29.480464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bae67-c301-42e2-a040-4d951dcffdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:29.481351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bae67-c301-42e2-a040-4d951dcffdfb admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:29.481688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bae67-c301-42e2-a040-4d951dcffdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:29.481916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bae67-c301-42e2-a040-4d951dcffdfb admin admin] PATH_INFO: `/v3/projects/7c5aa0685fb8459382985c4fceb4cb50/users/a65001615a10420db90c9840d2658fa0/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:29.501816 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bae67-c301-42e2-a040-4d951dcffdfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c5aa0685fb8459382985c4fceb4cb50, user_id=a65001615a10420db90c9840d2658fa0, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:29.505655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bae67-c301-42e2-a040-4d951dcffdfb admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:29.515849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:29.522179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 973/1929] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:29 2022] PUT /identity/v3/projects/7c5aa0685fb8459382985c4fceb4cb50/users/a65001615a10420db90c9840d2658fa0/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:29.523959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253dd1d-6e65-448c-9805-432c2bb58c2b None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:29.523959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253dd1d-6e65-448c-9805-432c2bb58c2b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:29.523959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253dd1d-6e65-448c-9805-432c2bb58c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:29.548761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e253dd1d-6e65-448c-9805-432c2bb58c2b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:29.550217 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e253dd1d-6e65-448c-9805-432c2bb58c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:29.558282 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7da19-89b9-4b89-98ef-13d3ff7e2743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:29.560161 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7da19-89b9-4b89-98ef-13d3ff7e2743 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:29.571243 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 957/1930] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:29.572789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fcfee-c952-409b-962a-72f20beaf936 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:29.573222 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fcfee-c952-409b-962a-72f20beaf936 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:29.573568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fcfee-c952-409b-962a-72f20beaf936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:29.608901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6a1fcfee-c952-409b-962a-72f20beaf936 None None] MFA Rules not processed for user `a65001615a10420db90c9840d2658fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:29.610736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6a1fcfee-c952-409b-962a-72f20beaf936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:29.634897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 974/1931] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:29.640305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253dd1d-6e65-448c-9805-432c2bb58c2b None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:29.652327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253dd1d-6e65-448c-9805-432c2bb58c2b None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:29.660337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12e943-926d-4de9-8661-559a1ee70a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:29.661859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12e943-926d-4de9-8661-559a1ee70a7f admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:29.662561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12e943-926d-4de9-8661-559a1ee70a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:29.662997 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12e943-926d-4de9-8661-559a1ee70a7f admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:29.663811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12e943-926d-4de9-8661-559a1ee70a7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:29.664303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12e943-926d-4de9-8661-559a1ee70a7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:29.666287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12e943-926d-4de9-8661-559a1ee70a7f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:29.691989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 975/1932] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:55:29.692946 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12e943-926d-4de9-8661-559a1ee70a7f admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:29.709000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12e943-926d-4de9-8661-559a1ee70a7f admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:29.715571 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730feaee-da69-4beb-b6e6-31eb56ab6bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:29.716297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730feaee-da69-4beb-b6e6-31eb56ab6bfe admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:29.716696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730feaee-da69-4beb-b6e6-31eb56ab6bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:29.717718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730feaee-da69-4beb-b6e6-31eb56ab6bfe admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:29.718440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730feaee-da69-4beb-b6e6-31eb56ab6bfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:29.721528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730feaee-da69-4beb-b6e6-31eb56ab6bfe admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:29.768710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 976/1933] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:29 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:29.769508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730feaee-da69-4beb-b6e6-31eb56ab6bfe admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:29.800795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 958/1934] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:29.801920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fcfee-c952-409b-962a-72f20beaf936 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:29.817564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fcfee-c952-409b-962a-72f20beaf936 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:29.834206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:29.930308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95bd75-4e7d-4262-a39e-8f60747f9c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:29.931832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95bd75-4e7d-4262-a39e-8f60747f9c5a admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:29.932143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95bd75-4e7d-4262-a39e-8f60747f9c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:29.932413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95bd75-4e7d-4262-a39e-8f60747f9c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:29.933181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95bd75-4e7d-4262-a39e-8f60747f9c5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:29.936456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95bd75-4e7d-4262-a39e-8f60747f9c5a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:29.940068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730feaee-da69-4beb-b6e6-31eb56ab6bfe admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:29.954130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202f71c-3045-47a7-9479-1f6a2c6b9cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:29.955126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202f71c-3045-47a7-9479-1f6a2c6b9cab admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:29.955508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202f71c-3045-47a7-9479-1f6a2c6b9cab admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:29.955875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202f71c-3045-47a7-9479-1f6a2c6b9cab admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:29.956732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202f71c-3045-47a7-9479-1f6a2c6b9cab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:29.958897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202f71c-3045-47a7-9479-1f6a2c6b9cab admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:29.998525 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 959/1935] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:29 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:30.002322 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95bd75-4e7d-4262-a39e-8f60747f9c5a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.019784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 977/1936] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:29 2022] POST /identity/v3/users => generated 283 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:30.034530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202f71c-3045-47a7-9479-1f6a2c6b9cab admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.065421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202f71c-3045-47a7-9479-1f6a2c6b9cab admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.145824 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:30.199070 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95bd75-4e7d-4262-a39e-8f60747f9c5a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.206534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bab92-e5d1-45d2-80bd-71478f35368d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.207484 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bab92-e5d1-45d2-80bd-71478f35368d admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.207778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bab92-e5d1-45d2-80bd-71478f35368d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.208226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bab92-e5d1-45d2-80bd-71478f35368d admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.208878 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bab92-e5d1-45d2-80bd-71478f35368d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.211309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bab92-e5d1-45d2-80bd-71478f35368d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.215779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb8b2d-4f95-42ca-95e5-e7655c8e8fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.216700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb8b2d-4f95-42ca-95e5-e7655c8e8fcf admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.217213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb8b2d-4f95-42ca-95e5-e7655c8e8fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.217602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb8b2d-4f95-42ca-95e5-e7655c8e8fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.218470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb8b2d-4f95-42ca-95e5-e7655c8e8fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.221115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb8b2d-4f95-42ca-95e5-e7655c8e8fcf admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.242450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 978/1937] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:30.242631 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 960/1938] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:30 2022] POST /identity/v3/users => generated 299 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:55:30.243938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bab92-e5d1-45d2-80bd-71478f35368d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.244051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb8b2d-4f95-42ca-95e5-e7655c8e8fcf admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.257311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bab92-e5d1-45d2-80bd-71478f35368d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.267476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a364b-0d38-4c61-890b-d6c622d172e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.268102 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb8b2d-4f95-42ca-95e5-e7655c8e8fcf admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.268440 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a364b-0d38-4c61-890b-d6c622d172e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.268815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a364b-0d38-4c61-890b-d6c622d172e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.269183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a364b-0d38-4c61-890b-d6c622d172e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.270041 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a364b-0d38-4c61-890b-d6c622d172e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.272367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a364b-0d38-4c61-890b-d6c622d172e1 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.272791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e47461-1aae-4d0f-8510-9264da3cbbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.274678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e47461-1aae-4d0f-8510-9264da3cbbd4 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.275026 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e47461-1aae-4d0f-8510-9264da3cbbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.275300 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e47461-1aae-4d0f-8510-9264da3cbbd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.276044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e47461-1aae-4d0f-8510-9264da3cbbd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.280064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e47461-1aae-4d0f-8510-9264da3cbbd4 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.289226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 961/1939] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:30.290540 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a364b-0d38-4c61-890b-d6c622d172e1 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.306673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 979/1940] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:30.306918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a364b-0d38-4c61-890b-d6c622d172e1 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.313536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffcc1c-8053-4748-a4bd-ae8649badfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.314576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffcc1c-8053-4748-a4bd-ae8649badfa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.315574 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffcc1c-8053-4748-a4bd-ae8649badfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.316001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffcc1c-8053-4748-a4bd-ae8649badfa2 admin admin] PATH_INFO: `/v3/projects/473e38572d3d416a9e558c4d748d155f/users/9a81c18815754c3e9b7a864a06867777/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.317069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e47461-1aae-4d0f-8510-9264da3cbbd4 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.333911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e47461-1aae-4d0f-8510-9264da3cbbd4 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.340119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83221e87-c873-4f98-8277-f4cf7469aba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.341249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83221e87-c873-4f98-8277-f4cf7469aba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.341659 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83221e87-c873-4f98-8277-f4cf7469aba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.342193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83221e87-c873-4f98-8277-f4cf7469aba5 admin admin] PATH_INFO: `/v3/projects/a648a45e39d64e098cb3c98ad989f25c/users/bb9d082af9ee4372bc80976a6c9e6978/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.356379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffcc1c-8053-4748-a4bd-ae8649badfa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=473e38572d3d416a9e558c4d748d155f, user_id=9a81c18815754c3e9b7a864a06867777, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.363519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffcc1c-8053-4748-a4bd-ae8649badfa2 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.375019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83221e87-c873-4f98-8277-f4cf7469aba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a648a45e39d64e098cb3c98ad989f25c, user_id=bb9d082af9ee4372bc80976a6c9e6978, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.381379 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83221e87-c873-4f98-8277-f4cf7469aba5 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.391511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 962/1941] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:30 2022] PUT /identity/v3/projects/473e38572d3d416a9e558c4d748d155f/users/9a81c18815754c3e9b7a864a06867777/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:30.407758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7aae2-c37d-47a2-bb02-58a9d851374d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.408148 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7aae2-c37d-47a2-bb02-58a9d851374d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.411223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7aae2-c37d-47a2-bb02-58a9d851374d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.429569 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 980/1942] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:30 2022] PUT /identity/v3/projects/a648a45e39d64e098cb3c98ad989f25c/users/bb9d082af9ee4372bc80976a6c9e6978/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:30.432822 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83221e87-c873-4f98-8277-f4cf7469aba5 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.453332 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-37a7aae2-c37d-47a2-bb02-58a9d851374d None None] MFA Rules not processed for user `9a81c18815754c3e9b7a864a06867777`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:30.455231 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-37a7aae2-c37d-47a2-bb02-58a9d851374d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:30.537390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83221e87-c873-4f98-8277-f4cf7469aba5 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.544667 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2f0c9-86d7-4ddb-8ae4-5c2dff7f4aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.546154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2f0c9-86d7-4ddb-8ae4-5c2dff7f4aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.546457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2f0c9-86d7-4ddb-8ae4-5c2dff7f4aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.546730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2f0c9-86d7-4ddb-8ae4-5c2dff7f4aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.547650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2f0c9-86d7-4ddb-8ae4-5c2dff7f4aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.549344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2f0c9-86d7-4ddb-8ae4-5c2dff7f4aa6 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.556453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 963/1943] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:30.560863 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7aae2-c37d-47a2-bb02-58a9d851374d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.576808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 981/1944] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:55:30.579043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2f0c9-86d7-4ddb-8ae4-5c2dff7f4aa6 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.586067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:30.586993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7aae2-c37d-47a2-bb02-58a9d851374d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.592224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2f0c9-86d7-4ddb-8ae4-5c2dff7f4aa6 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.593408 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcf9e7-f22a-43c6-93b4-c70158aad5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.594211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcf9e7-f22a-43c6-93b4-c70158aad5ff admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.594592 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcf9e7-f22a-43c6-93b4-c70158aad5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.595006 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcf9e7-f22a-43c6-93b4-c70158aad5ff admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.595966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdcf9e7-f22a-43c6-93b4-c70158aad5ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.599413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdcf9e7-f22a-43c6-93b4-c70158aad5ff admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.608706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9997f-5e1c-4a0e-955c-1ad9d3b23592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.609619 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9997f-5e1c-4a0e-955c-1ad9d3b23592 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.614609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9997f-5e1c-4a0e-955c-1ad9d3b23592 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.614773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9997f-5e1c-4a0e-955c-1ad9d3b23592 admin admin] PATH_INFO: `/v3/projects/a648a45e39d64e098cb3c98ad989f25c/users/bb9d082af9ee4372bc80976a6c9e6978/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.618439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9997f-5e1c-4a0e-955c-1ad9d3b23592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a648a45e39d64e098cb3c98ad989f25c, user_id=bb9d082af9ee4372bc80976a6c9e6978, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.621465 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9997f-5e1c-4a0e-955c-1ad9d3b23592 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.639975 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 964/1945] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:30 2022] POST /identity/v3/projects => generated 363 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:30.643793 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 982/1946] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:30 2022] PUT /identity/v3/projects/a648a45e39d64e098cb3c98ad989f25c/users/bb9d082af9ee4372bc80976a6c9e6978/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:30.647556 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcf9e7-f22a-43c6-93b4-c70158aad5ff admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.655195 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46eda7-bdb5-4807-877b-5080961688eb None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.655683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46eda7-bdb5-4807-877b-5080961688eb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.656074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46eda7-bdb5-4807-877b-5080961688eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.687229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-fd46eda7-bdb5-4807-877b-5080961688eb None None] MFA Rules not processed for user `bb9d082af9ee4372bc80976a6c9e6978`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:30.688517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-fd46eda7-bdb5-4807-877b-5080961688eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:30.731206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcf9e7-f22a-43c6-93b4-c70158aad5ff admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.738223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6dc35-b4cc-4915-83e2-c80baf154c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.739277 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6dc35-b4cc-4915-83e2-c80baf154c23 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.739563 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6dc35-b4cc-4915-83e2-c80baf154c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.739852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6dc35-b4cc-4915-83e2-c80baf154c23 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.740804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6dc35-b4cc-4915-83e2-c80baf154c23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.743761 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6dc35-b4cc-4915-83e2-c80baf154c23 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.770483 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:30.785007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 965/1947] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:30 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:30.785785 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 983/1948] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:30.789070 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6dc35-b4cc-4915-83e2-c80baf154c23 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.796372 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46eda7-bdb5-4807-877b-5080961688eb None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.797740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-fd46eda7-bdb5-4807-877b-5080961688eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:30.799427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6dc35-b4cc-4915-83e2-c80baf154c23 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.804049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46e878-d88d-4754-a6ef-a4d8fdc43195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.804717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46e878-d88d-4754-a6ef-a4d8fdc43195 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.804998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46e878-d88d-4754-a6ef-a4d8fdc43195 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.805270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46e878-d88d-4754-a6ef-a4d8fdc43195 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.805987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46e878-d88d-4754-a6ef-a4d8fdc43195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.807614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46e878-d88d-4754-a6ef-a4d8fdc43195 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.818011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 966/1949] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:55:30.822078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46e878-d88d-4754-a6ef-a4d8fdc43195 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.833068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46e878-d88d-4754-a6ef-a4d8fdc43195 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.837847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca74b9-32b1-4bdd-8f7e-703ff5e09651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.839098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca74b9-32b1-4bdd-8f7e-703ff5e09651 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.839098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca74b9-32b1-4bdd-8f7e-703ff5e09651 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.839365 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca74b9-32b1-4bdd-8f7e-703ff5e09651 admin admin] PATH_INFO: `/v3/projects/ec4f932b13024ba5a991d853e8a136d8/users/f8955b8b412c42ae8df1384c6d94dece/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.854930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca74b9-32b1-4bdd-8f7e-703ff5e09651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4f932b13024ba5a991d853e8a136d8, user_id=f8955b8b412c42ae8df1384c6d94dece, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.856425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46eda7-bdb5-4807-877b-5080961688eb None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.857891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca74b9-32b1-4bdd-8f7e-703ff5e09651 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.860928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022ddc6-f22e-425f-8bd8-0a077eb07d10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.861680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022ddc6-f22e-425f-8bd8-0a077eb07d10 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.861969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022ddc6-f22e-425f-8bd8-0a077eb07d10 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.862258 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022ddc6-f22e-425f-8bd8-0a077eb07d10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.871459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 967/1950] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:30 2022] PUT /identity/v3/projects/ec4f932b13024ba5a991d853e8a136d8/users/f8955b8b412c42ae8df1384c6d94dece/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:30.871743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022ddc6-f22e-425f-8bd8-0a077eb07d10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.873327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022ddc6-f22e-425f-8bd8-0a077eb07d10 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.875337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca74b9-32b1-4bdd-8f7e-703ff5e09651 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.883764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 984/1951] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:55:30.940288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca74b9-32b1-4bdd-8f7e-703ff5e09651 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.944883 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c0202-f24e-4095-a472-35b564f4badd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.945547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c0202-f24e-4095-a472-35b564f4badd admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.945828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c0202-f24e-4095-a472-35b564f4badd admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.946102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c0202-f24e-4095-a472-35b564f4badd admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.946845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c0202-f24e-4095-a472-35b564f4badd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.948451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c0202-f24e-4095-a472-35b564f4badd admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.959596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 968/1952] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:55:30.963965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022ddc6-f22e-425f-8bd8-0a077eb07d10 service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:30.975312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022ddc6-f22e-425f-8bd8-0a077eb07d10 service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:30.979919 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8228c0-59ad-4f6d-8493-02191ac0a5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:30.981185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8228c0-59ad-4f6d-8493-02191ac0a5b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:30.981474 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8228c0-59ad-4f6d-8493-02191ac0a5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:30.982696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8228c0-59ad-4f6d-8493-02191ac0a5b4 admin admin] PATH_INFO: `/v3/projects/ec4f932b13024ba5a991d853e8a136d8/users/f8955b8b412c42ae8df1384c6d94dece/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:30.984897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8228c0-59ad-4f6d-8493-02191ac0a5b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4f932b13024ba5a991d853e8a136d8, user_id=f8955b8b412c42ae8df1384c6d94dece, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:30.987617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8228c0-59ad-4f6d-8493-02191ac0a5b4 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:30.999240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 985/1953] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:30 2022] PUT /identity/v3/projects/ec4f932b13024ba5a991d853e8a136d8/users/f8955b8b412c42ae8df1384c6d94dece/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:31.009958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16f471-2288-4d04-a9c4-544c776d727e None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:31.010332 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16f471-2288-4d04-a9c4-544c776d727e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:31.010682 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16f471-2288-4d04-a9c4-544c776d727e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:31.041682 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-1a16f471-2288-4d04-a9c4-544c776d727e None None] MFA Rules not processed for user `f8955b8b412c42ae8df1384c6d94dece`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:31.043035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1a16f471-2288-4d04-a9c4-544c776d727e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:31.136644 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 969/1954] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:31 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:31.149824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8228c0-59ad-4f6d-8493-02191ac0a5b4 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:31.229167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8228c0-59ad-4f6d-8493-02191ac0a5b4 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:31.239647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9edaf6b-4600-4055-87c2-30a4e9c87987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:31.240470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edaf6b-4600-4055-87c2-30a4e9c87987 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:31.240874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edaf6b-4600-4055-87c2-30a4e9c87987 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:31.241205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edaf6b-4600-4055-87c2-30a4e9c87987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:31.271693 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9edaf6b-4600-4055-87c2-30a4e9c87987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:31.273434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9edaf6b-4600-4055-87c2-30a4e9c87987 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:31.286092 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 986/1955] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:31 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:31.637837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16f471-2288-4d04-a9c4-544c776d727e None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:31.687442 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:31.707152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16f471-2288-4d04-a9c4-544c776d727e None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:31.712318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b5604-e9b2-4fae-8641-79808cc51b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:31.713289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b5604-e9b2-4fae-8641-79808cc51b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:31.713578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b5604-e9b2-4fae-8641-79808cc51b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:31.713985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b5604-e9b2-4fae-8641-79808cc51b09 admin admin] PATH_INFO: `/v3/users/a65001615a10420db90c9840d2658fa0` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:31.715309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b5604-e9b2-4fae-8641-79808cc51b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a65001615a10420db90c9840d2658fa0)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:31.718366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b5604-e9b2-4fae-8641-79808cc51b09 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:31.776673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-761b5604-e9b2-4fae-8641-79808cc51b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a65001615a10420db90c9840d2658fa0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:31.787853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-761b5604-e9b2-4fae-8641-79808cc51b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a65001615a10420db90c9840d2658fa0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:31.788990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-761b5604-e9b2-4fae-8641-79808cc51b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a65001615a10420db90c9840d2658fa0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:31.801073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-761b5604-e9b2-4fae-8641-79808cc51b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a65001615a10420db90c9840d2658fa0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:31.818305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9edaf6b-4600-4055-87c2-30a4e9c87987 service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:31.825326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-761b5604-e9b2-4fae-8641-79808cc51b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a65001615a10420db90c9840d2658fa0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:31.833121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9edaf6b-4600-4055-87c2-30a4e9c87987 service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:31.834586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 970/1956] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:31 2022] DELETE /identity/v3/users/a65001615a10420db90c9840d2658fa0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:31.839043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e21397-b26d-44a1-9f06-ef367b983273 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:31.839393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e21397-b26d-44a1-9f06-ef367b983273 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:31.839731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e21397-b26d-44a1-9f06-ef367b983273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:31.843516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:31.859448 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-47e21397-b26d-44a1-9f06-ef367b983273 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:31.860750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-47e21397-b26d-44a1-9f06-ef367b983273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:31.914277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230856b8-3fb8-4e2d-94fd-4374d4356fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:31.915576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230856b8-3fb8-4e2d-94fd-4374d4356fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:31.915962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230856b8-3fb8-4e2d-94fd-4374d4356fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:31.916422 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230856b8-3fb8-4e2d-94fd-4374d4356fc3 admin admin] PATH_INFO: `/v3/users/9a81c18815754c3e9b7a864a06867777` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:31.918153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230856b8-3fb8-4e2d-94fd-4374d4356fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a81c18815754c3e9b7a864a06867777)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:31.921145 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230856b8-3fb8-4e2d-94fd-4374d4356fc3 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:31.930957 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 971/1957] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:31.941181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e21397-b26d-44a1-9f06-ef367b983273 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:31.944458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-47e21397-b26d-44a1-9f06-ef367b983273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:31.979580 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-230856b8-3fb8-4e2d-94fd-4374d4356fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a81c18815754c3e9b7a864a06867777'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:31.989427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-230856b8-3fb8-4e2d-94fd-4374d4356fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a81c18815754c3e9b7a864a06867777'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:31.999270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-230856b8-3fb8-4e2d-94fd-4374d4356fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a81c18815754c3e9b7a864a06867777'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:32.000335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-230856b8-3fb8-4e2d-94fd-4374d4356fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a81c18815754c3e9b7a864a06867777'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:32.001079 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-230856b8-3fb8-4e2d-94fd-4374d4356fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a81c18815754c3e9b7a864a06867777'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:32.011151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e21397-b26d-44a1-9f06-ef367b983273 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:32.015954 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c067438-4f2e-40a4-8858-b57f2530020f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:32.016692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c067438-4f2e-40a4-8858-b57f2530020f service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:32.017149 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c067438-4f2e-40a4-8858-b57f2530020f service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:32.017392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c067438-4f2e-40a4-8858-b57f2530020f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:32.019275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9c067438-4f2e-40a4-8858-b57f2530020f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:32.026022 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 987/1958] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:31 2022] DELETE /identity/v3/users/9a81c18815754c3e9b7a864a06867777 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:32.030678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4e4c6-5fe7-4caa-ab9f-4e55c3a0d075 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:32.031159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4e4c6-5fe7-4caa-ab9f-4e55c3a0d075 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:32.031600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4e4c6-5fe7-4caa-ab9f-4e55c3a0d075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:32.041701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c067438-4f2e-40a4-8858-b57f2530020f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:32.043185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c067438-4f2e-40a4-8858-b57f2530020f service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:32.054040 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 972/1959] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:55:32.057757 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-67e4e4c6-5fe7-4caa-ab9f-4e55c3a0d075 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:32.059003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-67e4e4c6-5fe7-4caa-ab9f-4e55c3a0d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:32.120899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 988/1960] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:32.132214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c067438-4f2e-40a4-8858-b57f2530020f service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:32.133627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9c067438-4f2e-40a4-8858-b57f2530020f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:32.141375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:32.144902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c067438-4f2e-40a4-8858-b57f2530020f service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:32.203079 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb87a6-cf17-47a4-9825-4dc3c820ac80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:32.203752 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb87a6-cf17-47a4-9825-4dc3c820ac80 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:32.204060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb87a6-cf17-47a4-9825-4dc3c820ac80 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:32.204361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb87a6-cf17-47a4-9825-4dc3c820ac80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:32.215327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb87a6-cf17-47a4-9825-4dc3c820ac80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:32.217825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb87a6-cf17-47a4-9825-4dc3c820ac80 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:32.228153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 973/1961] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:32.589810 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4e4c6-5fe7-4caa-ab9f-4e55c3a0d075 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:32.591270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-67e4e4c6-5fe7-4caa-ab9f-4e55c3a0d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:32.617059 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4e4c6-5fe7-4caa-ab9f-4e55c3a0d075 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:32.635626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:32.636646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:32.637037 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:32.637482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] PATH_INFO: `/v3/projects/7c5aa0685fb8459382985c4fceb4cb50` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:32.639799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c5aa0685fb8459382985c4fceb4cb50)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:32.642676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:32.690805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:32.730270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c5aa0685fb8459382985c4fceb4cb50', 'request_id': 'req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:32.737226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c5aa0685fb8459382985c4fceb4cb50', 'request_id': 'req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:32.745532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c5aa0685fb8459382985c4fceb4cb50', 'request_id': 'req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:32.765596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] The token cache is being invalidate because project 7c5aa0685fb8459382985c4fceb4cb50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:32.766718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:32.768827 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:32.771248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 989/1962] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:32 2022] DELETE /identity/v3/projects/7c5aa0685fb8459382985c4fceb4cb50 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:32.775380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb87a6-cf17-47a4-9825-4dc3c820ac80 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:32.791722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:32.793445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb87a6-cf17-47a4-9825-4dc3c820ac80 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:32.795058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:32.911309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529ea0e-c1e7-4e4a-b504-4b5e77ba32f6 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:32.915150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:32.916154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:32.916517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:32.916894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:32.917109 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] PATH_INFO: `/v3/users/bb9d082af9ee4372bc80976a6c9e6978` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:32.917894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:32.918250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:32.918521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb9d082af9ee4372bc80976a6c9e6978)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:32.918716 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] PATH_INFO: `/v3/projects/473e38572d3d416a9e558c4d748d155f` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:32.921401 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] RBAC: Authorizing `identity:delete_project(project_id=473e38572d3d416a9e558c4d748d155f)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:32.921569 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:32.923457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:32.976619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb9d082af9ee4372bc80976a6c9e6978'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:32.984966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb9d082af9ee4372bc80976a6c9e6978'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:32.985383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '473e38572d3d416a9e558c4d748d155f', 'request_id': 'req-8cda2761-1489-4a4d-b65b-213c9d15babd'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:32.985672 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb9d082af9ee4372bc80976a6c9e6978'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:32.991739 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '473e38572d3d416a9e558c4d748d155f', 'request_id': 'req-8cda2761-1489-4a4d-b65b-213c9d15babd'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:32.995432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb9d082af9ee4372bc80976a6c9e6978'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.000879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '473e38572d3d416a9e558c4d748d155f', 'request_id': 'req-8cda2761-1489-4a4d-b65b-213c9d15babd'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.011116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb9d082af9ee4372bc80976a6c9e6978'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.016031 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] The token cache is being invalidate because project 473e38572d3d416a9e558c4d748d155f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:33.016347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.017634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.021279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 990/1963] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:32 2022] DELETE /identity/v3/projects/473e38572d3d416a9e558c4d748d155f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:33.026664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 974/1964] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:32 2022] DELETE /identity/v3/users/bb9d082af9ee4372bc80976a6c9e6978 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:33.026814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:33.031123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:33.135433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:33.162936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda2761-1489-4a4d-b65b-213c9d15babd admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:33.244476 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f39dd-8e42-423d-b712-323ab2508fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:33.245419 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f39dd-8e42-423d-b712-323ab2508fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:33.245710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f39dd-8e42-423d-b712-323ab2508fab admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:33.245991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f39dd-8e42-423d-b712-323ab2508fab admin admin] PATH_INFO: `/v3/users/f8955b8b412c42ae8df1384c6d94dece` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:33.247332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f39dd-8e42-423d-b712-323ab2508fab admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8955b8b412c42ae8df1384c6d94dece)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:33.250239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f39dd-8e42-423d-b712-323ab2508fab admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:33.305220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f21f39dd-8e42-423d-b712-323ab2508fab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8955b8b412c42ae8df1384c6d94dece'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.317084 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f21f39dd-8e42-423d-b712-323ab2508fab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8955b8b412c42ae8df1384c6d94dece'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.329230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f21f39dd-8e42-423d-b712-323ab2508fab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8955b8b412c42ae8df1384c6d94dece'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.330294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f21f39dd-8e42-423d-b712-323ab2508fab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8955b8b412c42ae8df1384c6d94dece'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.331089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f21f39dd-8e42-423d-b712-323ab2508fab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8955b8b412c42ae8df1384c6d94dece'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.358787 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:33.360486 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:33.374613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 991/1965] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:33 2022] DELETE /identity/v3/users/f8955b8b412c42ae8df1384c6d94dece => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:33.379837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8641c-f2d4-4043-8280-eb10141192bb admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:33.431374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:33.475540 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe180041-cea2-499a-97d2-f56044e5dbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:33.476529 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe180041-cea2-499a-97d2-f56044e5dbd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:33.476827 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe180041-cea2-499a-97d2-f56044e5dbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:33.477193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe180041-cea2-499a-97d2-f56044e5dbd0 admin admin] PATH_INFO: `/v3/projects/a648a45e39d64e098cb3c98ad989f25c` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:33.478505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe180041-cea2-499a-97d2-f56044e5dbd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a648a45e39d64e098cb3c98ad989f25c)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:33.480491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe180041-cea2-499a-97d2-f56044e5dbd0 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:33.532428 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fe180041-cea2-499a-97d2-f56044e5dbd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a648a45e39d64e098cb3c98ad989f25c', 'request_id': 'req-fe180041-cea2-499a-97d2-f56044e5dbd0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.538755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fe180041-cea2-499a-97d2-f56044e5dbd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a648a45e39d64e098cb3c98ad989f25c', 'request_id': 'req-fe180041-cea2-499a-97d2-f56044e5dbd0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.548513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fe180041-cea2-499a-97d2-f56044e5dbd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a648a45e39d64e098cb3c98ad989f25c', 'request_id': 'req-fe180041-cea2-499a-97d2-f56044e5dbd0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.563451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fe180041-cea2-499a-97d2-f56044e5dbd0 admin admin] The token cache is being invalidate because project a648a45e39d64e098cb3c98ad989f25c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:33.563781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fe180041-cea2-499a-97d2-f56044e5dbd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.564404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fe180041-cea2-499a-97d2-f56044e5dbd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:33.566194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 975/1966] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:33 2022] DELETE /identity/v3/projects/a648a45e39d64e098cb3c98ad989f25c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:33.571167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af692c-8caf-44aa-89f3-183169a853b3 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:33.571458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af692c-8caf-44aa-89f3-183169a853b3 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:33.571741 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af692c-8caf-44aa-89f3-183169a853b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:33.590531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-c3af692c-8caf-44aa-89f3-183169a853b3 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:33.592695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c3af692c-8caf-44aa-89f3-183169a853b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:33.671170 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 992/1967] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:33.682925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccb6bd-01b5-4ceb-af96-bc6acbc1ddb1 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:33.683201 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccb6bd-01b5-4ceb-af96-bc6acbc1ddb1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:33.683604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccb6bd-01b5-4ceb-af96-bc6acbc1ddb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:33.703511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-c1ccb6bd-01b5-4ceb-af96-bc6acbc1ddb1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:33.704706 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c1ccb6bd-01b5-4ceb-af96-bc6acbc1ddb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:33.712720 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 976/1968] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:33.717714 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af692c-8caf-44aa-89f3-183169a853b3 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:33.727912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af692c-8caf-44aa-89f3-183169a853b3 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:33.732804 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f7b68-61ac-49a4-adde-0f1358a139b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:33.733566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f7b68-61ac-49a4-adde-0f1358a139b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:33.733896 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f7b68-61ac-49a4-adde-0f1358a139b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:33.734258 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f7b68-61ac-49a4-adde-0f1358a139b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:33.735082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f7b68-61ac-49a4-adde-0f1358a139b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:33.735439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f7b68-61ac-49a4-adde-0f1358a139b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:33.737142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f7b68-61ac-49a4-adde-0f1358a139b0 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:33.739764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:33.752837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 993/1969] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:55:33.758163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccb6bd-01b5-4ceb-af96-bc6acbc1ddb1 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:33.776705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccb6bd-01b5-4ceb-af96-bc6acbc1ddb1 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:33.784897 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825ba07-b75e-4bea-ac99-5b75beac5d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:33.785653 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825ba07-b75e-4bea-ac99-5b75beac5d25 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:33.786010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825ba07-b75e-4bea-ac99-5b75beac5d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:33.786194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825ba07-b75e-4bea-ac99-5b75beac5d25 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:33.787105 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825ba07-b75e-4bea-ac99-5b75beac5d25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:33.792303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825ba07-b75e-4bea-ac99-5b75beac5d25 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:33.827865 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 977/1970] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:33 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:55:33.829098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f7b68-61ac-49a4-adde-0f1358a139b0 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:33.832927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825ba07-b75e-4bea-ac99-5b75beac5d25 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:33.834700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6825ba07-b75e-4bea-ac99-5b75beac5d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:33.841165 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f7b68-61ac-49a4-adde-0f1358a139b0 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:33.944432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825ba07-b75e-4bea-ac99-5b75beac5d25 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:33.953456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:33.954646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:33.955301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:33.955758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3 admin admin] PATH_INFO: `/v3/projects/ec4f932b13024ba5a991d853e8a136d8` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:33.957314 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec4f932b13024ba5a991d853e8a136d8)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:33.959262 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:33.983849 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef710cd-b3a8-4c98-9ff1-d21fd32d2464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:33.984679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef710cd-b3a8-4c98-9ff1-d21fd32d2464 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:33.985039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef710cd-b3a8-4c98-9ff1-d21fd32d2464 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:33.985391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef710cd-b3a8-4c98-9ff1-d21fd32d2464 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:33.986187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef710cd-b3a8-4c98-9ff1-d21fd32d2464 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:33.988532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef710cd-b3a8-4c98-9ff1-d21fd32d2464 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:34.007200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:34.025676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 994/1971] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:33 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:34.031637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef710cd-b3a8-4c98-9ff1-d21fd32d2464 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:34.032556 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec4f932b13024ba5a991d853e8a136d8', 'request_id': 'req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:34.042714 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec4f932b13024ba5a991d853e8a136d8', 'request_id': 'req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:34.042946 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef710cd-b3a8-4c98-9ff1-d21fd32d2464 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:34.052663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec4f932b13024ba5a991d853e8a136d8', 'request_id': 'req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:34.078745 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3 admin admin] The token cache is being invalidate because project ec4f932b13024ba5a991d853e8a136d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:34.079384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:34.081001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d1aa0ac9-65b9-4280-b7a5-61ea63c6d7a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:34.087377 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 978/1972] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:33 2022] DELETE /identity/v3/projects/ec4f932b13024ba5a991d853e8a136d8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:34.089083 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b614fc-19d2-4691-9fcd-2b0a71b1ff79 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.089379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b614fc-19d2-4691-9fcd-2b0a71b1ff79 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.089662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b614fc-19d2-4691-9fcd-2b0a71b1ff79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.115421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-71b614fc-19d2-4691-9fcd-2b0a71b1ff79 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:34.116648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-71b614fc-19d2-4691-9fcd-2b0a71b1ff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:34.131703 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033619ae-4292-4418-bd5f-da2b3f75fe6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:34.135123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033619ae-4292-4418-bd5f-da2b3f75fe6f admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.135123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033619ae-4292-4418-bd5f-da2b3f75fe6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.135123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033619ae-4292-4418-bd5f-da2b3f75fe6f admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.135123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033619ae-4292-4418-bd5f-da2b3f75fe6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:34.136492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033619ae-4292-4418-bd5f-da2b3f75fe6f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:34.152170 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 995/1973] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:34.153040 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033619ae-4292-4418-bd5f-da2b3f75fe6f admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:34.157541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-033619ae-4292-4418-bd5f-da2b3f75fe6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:34.204209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 979/1974] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:34.204209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b614fc-19d2-4691-9fcd-2b0a71b1ff79 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:34.207335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-71b614fc-19d2-4691-9fcd-2b0a71b1ff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:34.224325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033619ae-4292-4418-bd5f-da2b3f75fe6f admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:34.225014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b614fc-19d2-4691-9fcd-2b0a71b1ff79 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:34.229269 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f004fe-6c77-41ee-909d-0395301ac5f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:34.229922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f004fe-6c77-41ee-909d-0395301ac5f5 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.230205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f004fe-6c77-41ee-909d-0395301ac5f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.230484 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f004fe-6c77-41ee-909d-0395301ac5f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.234917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cb2cd-b0ea-4225-a36e-68827e524e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:34.235386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cb2cd-b0ea-4225-a36e-68827e524e59 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.235707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cb2cd-b0ea-4225-a36e-68827e524e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.235944 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cb2cd-b0ea-4225-a36e-68827e524e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.236705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cb2cd-b0ea-4225-a36e-68827e524e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:34.238341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cb2cd-b0ea-4225-a36e-68827e524e59 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:34.239432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d0f004fe-6c77-41ee-909d-0395301ac5f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:34.250154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 980/1975] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:34.251650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875afbb2-91a2-4127-aee8-b4c8ad9812da None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.251945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875afbb2-91a2-4127-aee8-b4c8ad9812da None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.252223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875afbb2-91a2-4127-aee8-b4c8ad9812da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.271167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-875afbb2-91a2-4127-aee8-b4c8ad9812da None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:34.272365 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-875afbb2-91a2-4127-aee8-b4c8ad9812da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:34.280878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:34.284480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 981/1976] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:34.284480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875afbb2-91a2-4127-aee8-b4c8ad9812da None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:34.291599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875afbb2-91a2-4127-aee8-b4c8ad9812da None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:34.297854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a733d35-9234-4053-afbf-3685ab0575b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:34.298748 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a733d35-9234-4053-afbf-3685ab0575b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.299062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a733d35-9234-4053-afbf-3685ab0575b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.299341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a733d35-9234-4053-afbf-3685ab0575b8 admin admin] PATH_INFO: `/v3/projects/8cd6cd96f64a43d79ed6c5178275a1a8/users/8fca4ff548a948d09976d7cdfa6ad902/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.315618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a733d35-9234-4053-afbf-3685ab0575b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd6cd96f64a43d79ed6c5178275a1a8, user_id=8fca4ff548a948d09976d7cdfa6ad902, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:34.316353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f004fe-6c77-41ee-909d-0395301ac5f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:34.318388 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a733d35-9234-4053-afbf-3685ab0575b8 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:34.319140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f004fe-6c77-41ee-909d-0395301ac5f5 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:34.332012 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 996/1977] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:34.332532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f004fe-6c77-41ee-909d-0395301ac5f5 service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:34.338611 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 982/1978] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:34 2022] PUT /identity/v3/projects/8cd6cd96f64a43d79ed6c5178275a1a8/users/8fca4ff548a948d09976d7cdfa6ad902/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:34.343686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f004fe-6c77-41ee-909d-0395301ac5f5 service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:34.344409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa089c-e22b-4b43-b2a7-4ec6190e4dcc None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.344800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa089c-e22b-4b43-b2a7-4ec6190e4dcc None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.345387 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa089c-e22b-4b43-b2a7-4ec6190e4dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.370532 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-f6fa089c-e22b-4b43-b2a7-4ec6190e4dcc None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:34.371937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f6fa089c-e22b-4b43-b2a7-4ec6190e4dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:34.431770 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a06751-12da-4127-8e68-0201a0809abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:34.432376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a06751-12da-4127-8e68-0201a0809abb admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.432727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a06751-12da-4127-8e68-0201a0809abb admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.433011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a06751-12da-4127-8e68-0201a0809abb admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.433742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a06751-12da-4127-8e68-0201a0809abb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:34.434129 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a06751-12da-4127-8e68-0201a0809abb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:34.437216 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a06751-12da-4127-8e68-0201a0809abb admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:34.452860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 997/1979] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:55:34.457288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a06751-12da-4127-8e68-0201a0809abb admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:34.469594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 983/1980] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:34.472337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a06751-12da-4127-8e68-0201a0809abb admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:34.480074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa089c-e22b-4b43-b2a7-4ec6190e4dcc None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:34.483166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f6fa089c-e22b-4b43-b2a7-4ec6190e4dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:34.486153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bc308-1ed9-4036-b5ea-7ba69d0b485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:34.486818 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bc308-1ed9-4036-b5ea-7ba69d0b485d admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.487353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bc308-1ed9-4036-b5ea-7ba69d0b485d admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.487649 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bc308-1ed9-4036-b5ea-7ba69d0b485d admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.488409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bc308-1ed9-4036-b5ea-7ba69d0b485d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:34.490821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bc308-1ed9-4036-b5ea-7ba69d0b485d admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:34.542448 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:34.545394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 998/1981] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:34 2022] POST /identity/v3/projects => generated 363 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:34.549656 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bc308-1ed9-4036-b5ea-7ba69d0b485d admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:34.616085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa089c-e22b-4b43-b2a7-4ec6190e4dcc None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:34.647755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bc308-1ed9-4036-b5ea-7ba69d0b485d admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:34.655104 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961ab69-b79f-4371-9d73-f9cf209e3d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:34.655792 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961ab69-b79f-4371-9d73-f9cf209e3d98 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.656074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961ab69-b79f-4371-9d73-f9cf209e3d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.656349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961ab69-b79f-4371-9d73-f9cf209e3d98 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.657230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961ab69-b79f-4371-9d73-f9cf209e3d98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:34.659226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961ab69-b79f-4371-9d73-f9cf209e3d98 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:34.691083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 999/1982] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:34 2022] POST /identity/v3/users => generated 282 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:34.695156 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961ab69-b79f-4371-9d73-f9cf209e3d98 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:34.710164 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961ab69-b79f-4371-9d73-f9cf209e3d98 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:34.718308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cd0c8-7b49-4d59-aac3-3fee289c006c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:34.719356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cd0c8-7b49-4d59-aac3-3fee289c006c admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.719833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cd0c8-7b49-4d59-aac3-3fee289c006c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.720284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cd0c8-7b49-4d59-aac3-3fee289c006c admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.721357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cd0c8-7b49-4d59-aac3-3fee289c006c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:34.723371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fba88-aa2a-43ed-a574-d701077f7425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:34.723789 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cd0c8-7b49-4d59-aac3-3fee289c006c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:34.724027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fba88-aa2a-43ed-a574-d701077f7425 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.724360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fba88-aa2a-43ed-a574-d701077f7425 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.724639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fba88-aa2a-43ed-a574-d701077f7425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.738929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fba88-aa2a-43ed-a574-d701077f7425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:34.740103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fba88-aa2a-43ed-a574-d701077f7425 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:34.744915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1000/1983] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:34.746468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cd0c8-7b49-4d59-aac3-3fee289c006c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:34.752651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 984/1984] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:34.785790 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cd0c8-7b49-4d59-aac3-3fee289c006c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:34.805861 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a3d02-1745-43da-a2ca-033187f246d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:34.806752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a3d02-1745-43da-a2ca-033187f246d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.807203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a3d02-1745-43da-a2ca-033187f246d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.807848 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a3d02-1745-43da-a2ca-033187f246d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.809069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a3d02-1745-43da-a2ca-033187f246d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:34.819041 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a3d02-1745-43da-a2ca-033187f246d3 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:34.840194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1001/1985] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:34.844883 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fba88-aa2a-43ed-a574-d701077f7425 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:34.856251 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fba88-aa2a-43ed-a574-d701077f7425 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:34.865430 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0dca65-7da8-459c-88b0-297729fea89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:34.866501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dca65-7da8-459c-88b0-297729fea89e admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.867846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dca65-7da8-459c-88b0-297729fea89e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.880484 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dca65-7da8-459c-88b0-297729fea89e admin admin] PATH_INFO: `/v3/projects/12e0c074192d45cc8056a5cbb526932b/users/8eb7ad081a324ea78983ec0674578d08/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:34.913855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0dca65-7da8-459c-88b0-297729fea89e admin admin] RBAC: Authorizing `identity:create_grant(project_id=12e0c074192d45cc8056a5cbb526932b, user_id=8eb7ad081a324ea78983ec0674578d08, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:34.917176 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0dca65-7da8-459c-88b0-297729fea89e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:34.941000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 985/1986] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:34 2022] PUT /identity/v3/projects/12e0c074192d45cc8056a5cbb526932b/users/8eb7ad081a324ea78983ec0674578d08/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:34.952101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0011e-59ad-4587-b51e-4559b1264ef9 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:34.952743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0011e-59ad-4587-b51e-4559b1264ef9 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:34.953051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0011e-59ad-4587-b51e-4559b1264ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:35.003971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-1ad0011e-59ad-4587-b51e-4559b1264ef9 None None] MFA Rules not processed for user `8eb7ad081a324ea78983ec0674578d08`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:35.005355 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1ad0011e-59ad-4587-b51e-4559b1264ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:35.033733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:35.125234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1002/1987] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:34 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:35.130193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0dca65-7da8-459c-88b0-297729fea89e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:35.238607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0dca65-7da8-459c-88b0-297729fea89e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:35.249060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dafca3-ab89-4429-83cc-97aa750ef2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:35.249993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dafca3-ab89-4429-83cc-97aa750ef2aa admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:35.250951 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dafca3-ab89-4429-83cc-97aa750ef2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:35.251300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dafca3-ab89-4429-83cc-97aa750ef2aa admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:35.252155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dafca3-ab89-4429-83cc-97aa750ef2aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:35.255396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dafca3-ab89-4429-83cc-97aa750ef2aa admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:35.291175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 986/1988] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:35 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:35.295815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0011e-59ad-4587-b51e-4559b1264ef9 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:35.309150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0011e-59ad-4587-b51e-4559b1264ef9 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:35.383589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e5e47-1cf2-4de9-a3ec-924f7c550bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:35.384405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e5e47-1cf2-4de9-a3ec-924f7c550bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:35.384777 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e5e47-1cf2-4de9-a3ec-924f7c550bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:35.385131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e5e47-1cf2-4de9-a3ec-924f7c550bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:35.386001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e5e47-1cf2-4de9-a3ec-924f7c550bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:35.389073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e5e47-1cf2-4de9-a3ec-924f7c550bc1 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:35.422833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1003/1989] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:35 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:55:35.426995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dafca3-ab89-4429-83cc-97aa750ef2aa admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:35.441807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dafca3-ab89-4429-83cc-97aa750ef2aa admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:35.448143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fdd80-395e-42fd-b59f-6aa2694e9050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:35.448938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fdd80-395e-42fd-b59f-6aa2694e9050 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:35.449380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fdd80-395e-42fd-b59f-6aa2694e9050 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:35.449669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fdd80-395e-42fd-b59f-6aa2694e9050 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:35.450652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fdd80-395e-42fd-b59f-6aa2694e9050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:35.454188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fdd80-395e-42fd-b59f-6aa2694e9050 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:35.470423 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 987/1990] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:35.474615 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e5e47-1cf2-4de9-a3ec-924f7c550bc1 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:35.493676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e5e47-1cf2-4de9-a3ec-924f7c550bc1 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:35.521257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b57ae-9490-4fcd-a4bf-4e25ea0d0c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:35.524193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b57ae-9490-4fcd-a4bf-4e25ea0d0c38 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:35.524493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b57ae-9490-4fcd-a4bf-4e25ea0d0c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:35.524785 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b57ae-9490-4fcd-a4bf-4e25ea0d0c38 admin admin] PATH_INFO: `/v3/projects/ca436d7ffa234f88ae0a594b878ae260/users/10c7699a7e444af38f448f16d2bce038/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:35.566756 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b57ae-9490-4fcd-a4bf-4e25ea0d0c38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca436d7ffa234f88ae0a594b878ae260, user_id=10c7699a7e444af38f448f16d2bce038, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:35.570912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b57ae-9490-4fcd-a4bf-4e25ea0d0c38 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:35.587921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1004/1991] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:35 2022] PUT /identity/v3/projects/ca436d7ffa234f88ae0a594b878ae260/users/10c7699a7e444af38f448f16d2bce038/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:35.592652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fdd80-395e-42fd-b59f-6aa2694e9050 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:35.604764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fdd80-395e-42fd-b59f-6aa2694e9050 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:35.635902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:35.681878 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1edec6-a885-44f0-8239-eef5c5c6bb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:35.682795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1edec6-a885-44f0-8239-eef5c5c6bb67 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:35.683186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1edec6-a885-44f0-8239-eef5c5c6bb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:35.683531 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1edec6-a885-44f0-8239-eef5c5c6bb67 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:35.684303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1edec6-a885-44f0-8239-eef5c5c6bb67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:35.686524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1edec6-a885-44f0-8239-eef5c5c6bb67 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:35.698013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 988/1992] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:35.703149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b57ae-9490-4fcd-a4bf-4e25ea0d0c38 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:35.720795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b57ae-9490-4fcd-a4bf-4e25ea0d0c38 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:35.726474 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd1d30-ac2a-4fcd-9e9e-877d1bd0ba98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:35.727864 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd1d30-ac2a-4fcd-9e9e-877d1bd0ba98 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:35.728153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd1d30-ac2a-4fcd-9e9e-877d1bd0ba98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:35.728570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd1d30-ac2a-4fcd-9e9e-877d1bd0ba98 admin admin] PATH_INFO: `/v3/projects/ca436d7ffa234f88ae0a594b878ae260/users/10c7699a7e444af38f448f16d2bce038/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:35.739826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5259c9-ff11-44cb-8567-e38ef3081381 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:35.740125 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5259c9-ff11-44cb-8567-e38ef3081381 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:35.740404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5259c9-ff11-44cb-8567-e38ef3081381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:35.740616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd1d30-ac2a-4fcd-9e9e-877d1bd0ba98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca436d7ffa234f88ae0a594b878ae260, user_id=10c7699a7e444af38f448f16d2bce038, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:35.743885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd1d30-ac2a-4fcd-9e9e-877d1bd0ba98 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:35.770377 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1005/1993] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:35 2022] PUT /identity/v3/projects/ca436d7ffa234f88ae0a594b878ae260/users/10c7699a7e444af38f448f16d2bce038/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:35.774681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dee9b-cd17-4653-848f-e1476e783ca5 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:35.775158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dee9b-cd17-4653-848f-e1476e783ca5 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:35.777484 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dee9b-cd17-4653-848f-e1476e783ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:35.779112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6c5259c9-ff11-44cb-8567-e38ef3081381 None None] MFA Rules not processed for user `8fca4ff548a948d09976d7cdfa6ad902`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:35.780791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6c5259c9-ff11-44cb-8567-e38ef3081381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:35.820936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-cd6dee9b-cd17-4653-848f-e1476e783ca5 None None] MFA Rules not processed for user `10c7699a7e444af38f448f16d2bce038`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:35.823228 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-cd6dee9b-cd17-4653-848f-e1476e783ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:35.892154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:35.895899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 989/1994] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:35.902142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5259c9-ff11-44cb-8567-e38ef3081381 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:35.935769 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1006/1995] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:35 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:35.949003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6dee9b-cd17-4653-848f-e1476e783ca5 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:36.007963 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5259c9-ff11-44cb-8567-e38ef3081381 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:36.015857 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8c2ac-e988-4fea-b8b7-f549fbb3fc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:36.016770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8c2ac-e988-4fea-b8b7-f549fbb3fc78 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:36.017114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8c2ac-e988-4fea-b8b7-f549fbb3fc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:36.017450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8c2ac-e988-4fea-b8b7-f549fbb3fc78 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:36.018277 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8c2ac-e988-4fea-b8b7-f549fbb3fc78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:36.021746 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8c2ac-e988-4fea-b8b7-f549fbb3fc78 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:36.034108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6dee9b-cd17-4653-848f-e1476e783ca5 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:36.039210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582c873-eb9c-4bfe-a28a-ab6a29b8fd1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:36.039929 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582c873-eb9c-4bfe-a28a-ab6a29b8fd1a service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:36.040137 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582c873-eb9c-4bfe-a28a-ab6a29b8fd1a service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:36.040471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582c873-eb9c-4bfe-a28a-ab6a29b8fd1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:36.053238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582c873-eb9c-4bfe-a28a-ab6a29b8fd1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:36.055081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582c873-eb9c-4bfe-a28a-ab6a29b8fd1a service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:36.066604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 990/1996] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:35 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:36.069288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1007/1997] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:35 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:36.076455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8c2ac-e988-4fea-b8b7-f549fbb3fc78 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:36.148511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:36.179884 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582c873-eb9c-4bfe-a28a-ab6a29b8fd1a service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:36.189835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582c873-eb9c-4bfe-a28a-ab6a29b8fd1a service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:36.197796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8c2ac-e988-4fea-b8b7-f549fbb3fc78 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:36.204476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd925c-ce46-4189-b344-c8eb67c7a3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:36.205198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd925c-ce46-4189-b344-c8eb67c7a3df admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:36.205485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd925c-ce46-4189-b344-c8eb67c7a3df admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:36.205770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd925c-ce46-4189-b344-c8eb67c7a3df admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:36.206464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd925c-ce46-4189-b344-c8eb67c7a3df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:36.208415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd925c-ce46-4189-b344-c8eb67c7a3df admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:36.235099 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 991/1998] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:36 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:36.237038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd925c-ce46-4189-b344-c8eb67c7a3df admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:36.251050 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd925c-ce46-4189-b344-c8eb67c7a3df admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:36.256851 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632cfd4-2d4d-42b3-a8bf-416470b26754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:36.257296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67ce9b-9f39-4bf3-ab29-b205a36f67ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:36.257526 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632cfd4-2d4d-42b3-a8bf-416470b26754 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:36.257816 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632cfd4-2d4d-42b3-a8bf-416470b26754 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:36.258088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632cfd4-2d4d-42b3-a8bf-416470b26754 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:36.258233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67ce9b-9f39-4bf3-ab29-b205a36f67ef service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:36.258608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67ce9b-9f39-4bf3-ab29-b205a36f67ef service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:36.259433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632cfd4-2d4d-42b3-a8bf-416470b26754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:36.259546 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67ce9b-9f39-4bf3-ab29-b205a36f67ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:36.260429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632cfd4-2d4d-42b3-a8bf-416470b26754 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:36.274913 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 992/1999] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:55:36.279179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632cfd4-2d4d-42b3-a8bf-416470b26754 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:36.289575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632cfd4-2d4d-42b3-a8bf-416470b26754 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:36.294653 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4a26e-a84a-4a61-82c6-91cb443593f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:36.295786 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4a26e-a84a-4a61-82c6-91cb443593f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:36.296019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4a26e-a84a-4a61-82c6-91cb443593f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:36.296235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4a26e-a84a-4a61-82c6-91cb443593f9 admin admin] PATH_INFO: `/v3/projects/d219e5c6f71a47c1b17ff0f68bee44e4/users/e2f55e1762ac4af8b413db20d0c99f99/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:36.314759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4a26e-a84a-4a61-82c6-91cb443593f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d219e5c6f71a47c1b17ff0f68bee44e4, user_id=e2f55e1762ac4af8b413db20d0c99f99, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:36.317502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4a26e-a84a-4a61-82c6-91cb443593f9 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:36.334973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 993/2000] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:36 2022] PUT /identity/v3/projects/d219e5c6f71a47c1b17ff0f68bee44e4/users/e2f55e1762ac4af8b413db20d0c99f99/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:36.339207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4a26e-a84a-4a61-82c6-91cb443593f9 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:36.347779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67ce9b-9f39-4bf3-ab29-b205a36f67ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:36.349349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67ce9b-9f39-4bf3-ab29-b205a36f67ef service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:36.360848 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1008/2001] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:36 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:36.457238 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4a26e-a84a-4a61-82c6-91cb443593f9 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:36.463114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc564af-bfe6-4dc8-8fce-5212f5e3837d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:36.464005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc564af-bfe6-4dc8-8fce-5212f5e3837d admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:36.464378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc564af-bfe6-4dc8-8fce-5212f5e3837d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:36.464772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc564af-bfe6-4dc8-8fce-5212f5e3837d admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:36.465690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc564af-bfe6-4dc8-8fce-5212f5e3837d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:36.468246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc564af-bfe6-4dc8-8fce-5212f5e3837d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:36.481264 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 994/2002] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:36.486443 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67ce9b-9f39-4bf3-ab29-b205a36f67ef service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:36.517845 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67ce9b-9f39-4bf3-ab29-b205a36f67ef service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:36.546225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b947bc-acad-4b25-ba39-bd797310c35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:36.547649 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b947bc-acad-4b25-ba39-bd797310c35d admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:36.547937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b947bc-acad-4b25-ba39-bd797310c35d admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:36.548532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b947bc-acad-4b25-ba39-bd797310c35d admin admin] PATH_INFO: `/v3/projects/d219e5c6f71a47c1b17ff0f68bee44e4/users/e2f55e1762ac4af8b413db20d0c99f99/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:36.561850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b947bc-acad-4b25-ba39-bd797310c35d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d219e5c6f71a47c1b17ff0f68bee44e4, user_id=e2f55e1762ac4af8b413db20d0c99f99, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:36.568457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b947bc-acad-4b25-ba39-bd797310c35d admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:36.598231 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1009/2003] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:36 2022] PUT /identity/v3/projects/d219e5c6f71a47c1b17ff0f68bee44e4/users/e2f55e1762ac4af8b413db20d0c99f99/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:37.640157 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d1e77-cffb-4251-882f-843a5aa5cbd6 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:37.640704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d1e77-cffb-4251-882f-843a5aa5cbd6 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:37.640704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d1e77-cffb-4251-882f-843a5aa5cbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:37.703835 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-448d1e77-cffb-4251-882f-843a5aa5cbd6 None None] MFA Rules not processed for user `e2f55e1762ac4af8b413db20d0c99f99`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:37.706335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-448d1e77-cffb-4251-882f-843a5aa5cbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:37.837331 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b947bc-acad-4b25-ba39-bd797310c35d admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:37.839434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-b4b947bc-acad-4b25-ba39-bd797310c35d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:37.895845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 995/2004] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:37.911031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d1e77-cffb-4251-882f-843a5aa5cbd6 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:37.914879 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-448d1e77-cffb-4251-882f-843a5aa5cbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:37.923234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:37.942994 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:38.020872 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b947bc-acad-4b25-ba39-bd797310c35d admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:38.026849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d1e77-cffb-4251-882f-843a5aa5cbd6 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:38.031340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406dd9e-94a8-4254-9bb5-8f597421ec60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:38.031516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:38.032277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:38.032564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:38.032843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] PATH_INFO: `/v3/users/5d95fbc2347946398ec5b360b957bf07` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:38.033153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406dd9e-94a8-4254-9bb5-8f597421ec60 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:38.033646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406dd9e-94a8-4254-9bb5-8f597421ec60 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:38.034129 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406dd9e-94a8-4254-9bb5-8f597421ec60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:38.035531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d95fbc2347946398ec5b360b957bf07)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:38.040855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:38.133714 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d95fbc2347946398ec5b360b957bf07'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:38.146078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406dd9e-94a8-4254-9bb5-8f597421ec60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:38.147604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406dd9e-94a8-4254-9bb5-8f597421ec60 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:38.148650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d95fbc2347946398ec5b360b957bf07'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:38.161041 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d95fbc2347946398ec5b360b957bf07'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:38.162169 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d95fbc2347946398ec5b360b957bf07'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:38.163765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d95fbc2347946398ec5b360b957bf07'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:38.164921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 996/2005] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:38.186926 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1010/2006] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:37 2022] DELETE /identity/v3/users/5d95fbc2347946398ec5b360b957bf07 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:38.192228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854c982-9838-461b-b4ae-c3e1b42b2349 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:38.192528 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854c982-9838-461b-b4ae-c3e1b42b2349 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:38.192809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854c982-9838-461b-b4ae-c3e1b42b2349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:38.226996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-9854c982-9838-461b-b4ae-c3e1b42b2349 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:38.228622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9854c982-9838-461b-b4ae-c3e1b42b2349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:38.355612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 997/2007] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:38.366089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:38.367558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:38.463021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba13c2d-01fc-4ea3-8dc7-dcca937a9605 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:38.468916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4817af0-eb73-4fa6-9d48-bf8b72870907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:38.470400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4817af0-eb73-4fa6-9d48-bf8b72870907 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:38.470820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4817af0-eb73-4fa6-9d48-bf8b72870907 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:38.471797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4817af0-eb73-4fa6-9d48-bf8b72870907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:38.484771 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4817af0-eb73-4fa6-9d48-bf8b72870907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:38.486172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4817af0-eb73-4fa6-9d48-bf8b72870907 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:38.498893 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1011/2008] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:38.819442 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854c982-9838-461b-b4ae-c3e1b42b2349 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:38.820983 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9854c982-9838-461b-b4ae-c3e1b42b2349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:38.834271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854c982-9838-461b-b4ae-c3e1b42b2349 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:38.839645 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:38.840620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:38.840978 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:38.841750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] PATH_INFO: `/v3/projects/8494c98a48ed4496b058be30bb20541e` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:38.843835 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8494c98a48ed4496b058be30bb20541e)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:38.845771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:38.896293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8494c98a48ed4496b058be30bb20541e', 'request_id': 'req-f088eabb-c39b-451f-8c35-5612d6bf6a07'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:38.903000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8494c98a48ed4496b058be30bb20541e', 'request_id': 'req-f088eabb-c39b-451f-8c35-5612d6bf6a07'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:38.911495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8494c98a48ed4496b058be30bb20541e', 'request_id': 'req-f088eabb-c39b-451f-8c35-5612d6bf6a07'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:38.913737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:38.925473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] The token cache is being invalidate because project 8494c98a48ed4496b058be30bb20541e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:38.926180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:38.927237 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:38.929032 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 998/2009] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:38 2022] DELETE /identity/v3/projects/8494c98a48ed4496b058be30bb20541e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:38.933181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4817af0-eb73-4fa6-9d48-bf8b72870907 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:38.947928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4817af0-eb73-4fa6-9d48-bf8b72870907 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:38.993515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:39.009581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:39.010452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:39.010739 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:39.011046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] PATH_INFO: `/v3/users/cffeb4e38b774419be10efe71b54c147` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:39.012246 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=cffeb4e38b774419be10efe71b54c147)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:39.014835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:39.061660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cffeb4e38b774419be10efe71b54c147'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.069503 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cffeb4e38b774419be10efe71b54c147'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.077090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cffeb4e38b774419be10efe71b54c147'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.077727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cffeb4e38b774419be10efe71b54c147'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.078376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cffeb4e38b774419be10efe71b54c147'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.095708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1012/2010] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:38 2022] DELETE /identity/v3/users/cffeb4e38b774419be10efe71b54c147 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:39.429719 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:39.433246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:39.474964 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088eabb-c39b-451f-8c35-5612d6bf6a07 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:39.547277 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:39.548230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:39.548578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:39.549040 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] PATH_INFO: `/v3/projects/c8765c5c581645a181effaadc2f17354` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:39.550382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8765c5c581645a181effaadc2f17354)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:39.552411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:39.607347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8765c5c581645a181effaadc2f17354', 'request_id': 'req-9d22334e-96f7-4aa0-9c57-648835090c93'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.614125 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8765c5c581645a181effaadc2f17354', 'request_id': 'req-9d22334e-96f7-4aa0-9c57-648835090c93'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.621713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8765c5c581645a181effaadc2f17354', 'request_id': 'req-9d22334e-96f7-4aa0-9c57-648835090c93'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.635068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] The token cache is being invalidate because project c8765c5c581645a181effaadc2f17354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:39.635366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.636026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.637790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 999/2011] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:39 2022] DELETE /identity/v3/projects/c8765c5c581645a181effaadc2f17354 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:39.641727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:39.643285 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:39.660095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b84879-b3b2-4bbd-bcf0-c7c7b68187ee admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:39.707300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:39.708839 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:39.734065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:39.734989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:39.735306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:39.735550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] PATH_INFO: `/v3/users/a0f5fefbafb44577b5159f7fed92a3ff` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:39.736767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f5fefbafb44577b5159f7fed92a3ff)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:39.738748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:39.785987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22334e-96f7-4aa0-9c57-648835090c93 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:39.790281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cb0d6-90f6-47ad-a543-6e0324b8f5ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:39.791133 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cb0d6-90f6-47ad-a543-6e0324b8f5ed service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:39.791422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cb0d6-90f6-47ad-a543-6e0324b8f5ed service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:39.791889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cb0d6-90f6-47ad-a543-6e0324b8f5ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:39.793738 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-718cb0d6-90f6-47ad-a543-6e0324b8f5ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:39.798949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f5fefbafb44577b5159f7fed92a3ff'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.810700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f5fefbafb44577b5159f7fed92a3ff'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.820490 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f5fefbafb44577b5159f7fed92a3ff'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.821207 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f5fefbafb44577b5159f7fed92a3ff'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.821833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f5fefbafb44577b5159f7fed92a3ff'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:39.840326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:39.845011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:39.861468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1013/2012] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:39 2022] DELETE /identity/v3/users/a0f5fefbafb44577b5159f7fed92a3ff => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:39.956830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cb0d6-90f6-47ad-a543-6e0324b8f5ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:39.960134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cb0d6-90f6-47ad-a543-6e0324b8f5ed service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:39.988409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1000/2013] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:55:40.299770 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:40.301257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:40.439797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907f9db-979a-43e2-b66d-9fbabf05dd22 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:40.444796 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e72e8-986f-4439-bcbf-fea08559855b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:40.445765 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e72e8-986f-4439-bcbf-fea08559855b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:40.446117 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e72e8-986f-4439-bcbf-fea08559855b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:40.446486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e72e8-986f-4439-bcbf-fea08559855b admin admin] PATH_INFO: `/v3/projects/654a0dfeade24088aff933999c6452f7` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:40.447991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e72e8-986f-4439-bcbf-fea08559855b admin admin] RBAC: Authorizing `identity:delete_project(project_id=654a0dfeade24088aff933999c6452f7)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:40.449768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e72e8-986f-4439-bcbf-fea08559855b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:40.516016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2d4e72e8-986f-4439-bcbf-fea08559855b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654a0dfeade24088aff933999c6452f7', 'request_id': 'req-2d4e72e8-986f-4439-bcbf-fea08559855b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:40.521755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2d4e72e8-986f-4439-bcbf-fea08559855b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654a0dfeade24088aff933999c6452f7', 'request_id': 'req-2d4e72e8-986f-4439-bcbf-fea08559855b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:40.530436 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2d4e72e8-986f-4439-bcbf-fea08559855b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654a0dfeade24088aff933999c6452f7', 'request_id': 'req-2d4e72e8-986f-4439-bcbf-fea08559855b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:40.542970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2d4e72e8-986f-4439-bcbf-fea08559855b admin admin] The token cache is being invalidate because project 654a0dfeade24088aff933999c6452f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:40.543191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2d4e72e8-986f-4439-bcbf-fea08559855b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:40.543815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2d4e72e8-986f-4439-bcbf-fea08559855b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:40.546480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1014/2014] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:40 2022] DELETE /identity/v3/projects/654a0dfeade24088aff933999c6452f7 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:40.552743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57629c4-3fd7-4003-82eb-9c734ea4451e None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:40.553036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57629c4-3fd7-4003-82eb-9c734ea4451e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:40.553316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57629c4-3fd7-4003-82eb-9c734ea4451e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:40.573400 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-e57629c4-3fd7-4003-82eb-9c734ea4451e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:40.575076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e57629c4-3fd7-4003-82eb-9c734ea4451e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:40.644927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1001/2015] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:40.653467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03074c-f74a-47ea-8bb2-b899f55249bc None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:40.653815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03074c-f74a-47ea-8bb2-b899f55249bc None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:40.654595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03074c-f74a-47ea-8bb2-b899f55249bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:40.674649 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-1b03074c-f74a-47ea-8bb2-b899f55249bc None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:40.675922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1b03074c-f74a-47ea-8bb2-b899f55249bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:40.683991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1015/2016] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:40.689159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57629c4-3fd7-4003-82eb-9c734ea4451e None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:40.699794 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57629c4-3fd7-4003-82eb-9c734ea4451e None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:40.705051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1043442-188b-4406-bd90-80774bcf4a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:40.705721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043442-188b-4406-bd90-80774bcf4a57 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:40.706005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043442-188b-4406-bd90-80774bcf4a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:40.706303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043442-188b-4406-bd90-80774bcf4a57 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:40.707054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1043442-188b-4406-bd90-80774bcf4a57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:40.707360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1043442-188b-4406-bd90-80774bcf4a57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:40.708949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1043442-188b-4406-bd90-80774bcf4a57 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:40.725084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1002/2017] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:55:40.726275 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03074c-f74a-47ea-8bb2-b899f55249bc None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:40.736018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03074c-f74a-47ea-8bb2-b899f55249bc None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:40.742073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a86aa-8050-401b-85da-53bcd0946986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:40.743101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a86aa-8050-401b-85da-53bcd0946986 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:40.743393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a86aa-8050-401b-85da-53bcd0946986 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:40.743671 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a86aa-8050-401b-85da-53bcd0946986 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:40.744423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a86aa-8050-401b-85da-53bcd0946986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:40.746616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a86aa-8050-401b-85da-53bcd0946986 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:40.779639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1016/2018] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:40 2022] POST /identity/v3/projects => generated 407 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:40.781480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1043442-188b-4406-bd90-80774bcf4a57 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:40.792047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1043442-188b-4406-bd90-80774bcf4a57 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:40.855966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c020e-efd9-43e5-bb8b-8a37fb39242e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:40.856663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c020e-efd9-43e5-bb8b-8a37fb39242e admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:40.856986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c020e-efd9-43e5-bb8b-8a37fb39242e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:40.857275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c020e-efd9-43e5-bb8b-8a37fb39242e admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:40.858051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c020e-efd9-43e5-bb8b-8a37fb39242e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:40.860131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c020e-efd9-43e5-bb8b-8a37fb39242e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:40.889406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1003/2019] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:40 2022] POST /identity/v3/users => generated 304 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:40.890979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a86aa-8050-401b-85da-53bcd0946986 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:40.901270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a86aa-8050-401b-85da-53bcd0946986 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:40.908321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed8ad5-cafb-4a86-a4b2-568705323cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:40.909363 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed8ad5-cafb-4a86-a4b2-568705323cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:40.909756 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed8ad5-cafb-4a86-a4b2-568705323cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:40.910140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed8ad5-cafb-4a86-a4b2-568705323cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:40.912781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed8ad5-cafb-4a86-a4b2-568705323cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:40.916090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed8ad5-cafb-4a86-a4b2-568705323cf3 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:40.936752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1017/2020] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:40.937854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c020e-efd9-43e5-bb8b-8a37fb39242e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:40.952097 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c020e-efd9-43e5-bb8b-8a37fb39242e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:40.954360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ffaf5-4274-4285-89f0-542b84fa5eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:40.955052 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ffaf5-4274-4285-89f0-542b84fa5eca admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:40.955339 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ffaf5-4274-4285-89f0-542b84fa5eca admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:40.955639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ffaf5-4274-4285-89f0-542b84fa5eca admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:40.956359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ffaf5-4274-4285-89f0-542b84fa5eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:40.957938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ffaf5-4274-4285-89f0-542b84fa5eca admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:40.968612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1004/2021] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:40.972865 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed8ad5-cafb-4a86-a4b2-568705323cf3 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:40.984378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed8ad5-cafb-4a86-a4b2-568705323cf3 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:40.989370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ac01f-9aec-493b-a37f-1a65dba6cc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:40.990248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac01f-9aec-493b-a37f-1a65dba6cc6d admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:40.990535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac01f-9aec-493b-a37f-1a65dba6cc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:40.990813 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac01f-9aec-493b-a37f-1a65dba6cc6d admin admin] PATH_INFO: `/v3/projects/1e864348a7104604a3f73fe949ed8951/users/8d726f3617b24175a4aa6421e01ecb68/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:41.005740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ac01f-9aec-493b-a37f-1a65dba6cc6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e864348a7104604a3f73fe949ed8951, user_id=8d726f3617b24175a4aa6421e01ecb68, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:41.008396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ac01f-9aec-493b-a37f-1a65dba6cc6d admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:41.021998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1018/2022] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:40 2022] PUT /identity/v3/projects/1e864348a7104604a3f73fe949ed8951/users/8d726f3617b24175a4aa6421e01ecb68/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:41.034248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63198e3-bdf3-465d-b4d4-95dfce2a5c36 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:41.034538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63198e3-bdf3-465d-b4d4-95dfce2a5c36 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:41.034817 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63198e3-bdf3-465d-b4d4-95dfce2a5c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:41.061917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-e63198e3-bdf3-465d-b4d4-95dfce2a5c36 None None] MFA Rules not processed for user `8d726f3617b24175a4aa6421e01ecb68`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:41.063081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e63198e3-bdf3-465d-b4d4-95dfce2a5c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:41.089349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:41.129382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1005/2023] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:41 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:41.133862 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ac01f-9aec-493b-a37f-1a65dba6cc6d admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:41.147877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:41.198210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ac01f-9aec-493b-a37f-1a65dba6cc6d admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:41.204173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758eb49-4679-453d-98d5-3a3459592667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:41.204835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758eb49-4679-453d-98d5-3a3459592667 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:41.205120 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758eb49-4679-453d-98d5-3a3459592667 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:41.205410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758eb49-4679-453d-98d5-3a3459592667 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:41.206114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758eb49-4679-453d-98d5-3a3459592667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:41.208048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758eb49-4679-453d-98d5-3a3459592667 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:41.234351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1019/2024] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:41 2022] POST /identity/v3/projects => generated 409 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:41.238516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63198e3-bdf3-465d-b4d4-95dfce2a5c36 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:41.249543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63198e3-bdf3-465d-b4d4-95dfce2a5c36 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:41.308849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5ff91-95a2-4ff0-bfd7-91af456c028d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:41.309536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5ff91-95a2-4ff0-bfd7-91af456c028d admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:41.309826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5ff91-95a2-4ff0-bfd7-91af456c028d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:41.310280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5ff91-95a2-4ff0-bfd7-91af456c028d admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:41.311328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5ff91-95a2-4ff0-bfd7-91af456c028d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:41.313650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5ff91-95a2-4ff0-bfd7-91af456c028d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:41.333965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1006/2025] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:41 2022] POST /identity/v3/users => generated 311 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:41.337967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758eb49-4679-453d-98d5-3a3459592667 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:41.349185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758eb49-4679-453d-98d5-3a3459592667 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:41.353745 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33f472-036b-436c-bb16-5cba839a3ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:41.354398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33f472-036b-436c-bb16-5cba839a3ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:41.354686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33f472-036b-436c-bb16-5cba839a3ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:41.354996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33f472-036b-436c-bb16-5cba839a3ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:41.355718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33f472-036b-436c-bb16-5cba839a3ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:41.357308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33f472-036b-436c-bb16-5cba839a3ff2 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:41.367656 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1020/2026] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:41.371813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5ff91-95a2-4ff0-bfd7-91af456c028d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:41.381983 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5ff91-95a2-4ff0-bfd7-91af456c028d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:41.386802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdba2c-183a-4593-9aaf-96ec4b04018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:41.387750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdba2c-183a-4593-9aaf-96ec4b04018a admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:41.388549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdba2c-183a-4593-9aaf-96ec4b04018a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:41.388828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdba2c-183a-4593-9aaf-96ec4b04018a admin admin] PATH_INFO: `/v3/projects/ce1fa9ab40324ea0b6a888cb64e28bcf/users/f16f60ec9ee9421ca81a3c4a6e089273/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:41.405988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdba2c-183a-4593-9aaf-96ec4b04018a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1fa9ab40324ea0b6a888cb64e28bcf, user_id=f16f60ec9ee9421ca81a3c4a6e089273, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:41.408673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdba2c-183a-4593-9aaf-96ec4b04018a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:41.421158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1007/2027] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:41 2022] PUT /identity/v3/projects/ce1fa9ab40324ea0b6a888cb64e28bcf/users/f16f60ec9ee9421ca81a3c4a6e089273/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:41.425241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33f472-036b-436c-bb16-5cba839a3ff2 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:41.436982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33f472-036b-436c-bb16-5cba839a3ff2 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:41.496762 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92095238-12ba-4db8-8cb3-28a592142549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:41.497968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92095238-12ba-4db8-8cb3-28a592142549 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:41.498420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92095238-12ba-4db8-8cb3-28a592142549 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:41.498701 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92095238-12ba-4db8-8cb3-28a592142549 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:41.499484 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92095238-12ba-4db8-8cb3-28a592142549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:41.501075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92095238-12ba-4db8-8cb3-28a592142549 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:41.511029 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1021/2028] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:41.516013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdba2c-183a-4593-9aaf-96ec4b04018a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:41.526548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdba2c-183a-4593-9aaf-96ec4b04018a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:41.531469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0efec2-b69e-4e6e-8458-c6a6559cca8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:41.532358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0efec2-b69e-4e6e-8458-c6a6559cca8b admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:41.532644 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0efec2-b69e-4e6e-8458-c6a6559cca8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:41.532919 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0efec2-b69e-4e6e-8458-c6a6559cca8b admin admin] PATH_INFO: `/v3/projects/ce1fa9ab40324ea0b6a888cb64e28bcf/users/f16f60ec9ee9421ca81a3c4a6e089273/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:41.535015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0efec2-b69e-4e6e-8458-c6a6559cca8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1fa9ab40324ea0b6a888cb64e28bcf, user_id=f16f60ec9ee9421ca81a3c4a6e089273, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:41.537748 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0efec2-b69e-4e6e-8458-c6a6559cca8b admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:41.548515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1008/2029] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:41 2022] PUT /identity/v3/projects/ce1fa9ab40324ea0b6a888cb64e28bcf/users/f16f60ec9ee9421ca81a3c4a6e089273/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:41.559764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c42b2-79f8-495a-af73-96b61f13a6eb None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:41.560751 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c42b2-79f8-495a-af73-96b61f13a6eb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:41.561125 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c42b2-79f8-495a-af73-96b61f13a6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:41.587788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-991c42b2-79f8-495a-af73-96b61f13a6eb None None] MFA Rules not processed for user `f16f60ec9ee9421ca81a3c4a6e089273`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:41.588929 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-991c42b2-79f8-495a-af73-96b61f13a6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:41.656586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1022/2030] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:41 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:41.667499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0efec2-b69e-4e6e-8458-c6a6559cca8b admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:41.669203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-da0efec2-b69e-4e6e-8458-c6a6559cca8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:41.724338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0efec2-b69e-4e6e-8458-c6a6559cca8b admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:41.727946 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a1b1c-79a1-4fca-8e56-278dfc538547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:41.728583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a1b1c-79a1-4fca-8e56-278dfc538547 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:41.728864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a1b1c-79a1-4fca-8e56-278dfc538547 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:41.729134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a1b1c-79a1-4fca-8e56-278dfc538547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:41.738043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a1b1c-79a1-4fca-8e56-278dfc538547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:41.739497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a1b1c-79a1-4fca-8e56-278dfc538547 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:41.749306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1009/2031] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:41 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:41.898333 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c42b2-79f8-495a-af73-96b61f13a6eb None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:41.907839 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c42b2-79f8-495a-af73-96b61f13a6eb None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:41.911385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa877d-fd2b-4b67-b860-01bcb69c9b15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:41.912021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa877d-fd2b-4b67-b860-01bcb69c9b15 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:41.912303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa877d-fd2b-4b67-b860-01bcb69c9b15 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:41.912577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa877d-fd2b-4b67-b860-01bcb69c9b15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:41.917986 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:41.990938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa877d-fd2b-4b67-b860-01bcb69c9b15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:41.992388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa877d-fd2b-4b67-b860-01bcb69c9b15 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:42.002398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1023/2032] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:41 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:44.147252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2dbee-f1d2-481e-888d-2ad61512ff4f None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:44.147772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2dbee-f1d2-481e-888d-2ad61512ff4f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:44.147865 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2dbee-f1d2-481e-888d-2ad61512ff4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:44.165977 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-f0c2dbee-f1d2-481e-888d-2ad61512ff4f None None] MFA Rules not processed for user `a9a3080f9aef4323952eed19ef9bb074`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:44.167206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f0c2dbee-f1d2-481e-888d-2ad61512ff4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:44.183720 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:44.225005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1010/2033] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:45.629476 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa877d-fd2b-4b67-b860-01bcb69c9b15 service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:45.630995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-87fa877d-fd2b-4b67-b860-01bcb69c9b15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:45.704883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa877d-fd2b-4b67-b860-01bcb69c9b15 service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:45.709790 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07716b3a-59f3-4a8e-a834-34d44c22b12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:45.710673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07716b3a-59f3-4a8e-a834-34d44c22b12e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:45.710988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07716b3a-59f3-4a8e-a834-34d44c22b12e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:45.711270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07716b3a-59f3-4a8e-a834-34d44c22b12e admin admin] PATH_INFO: `/v3/users/fd88c8e9166940d8a09657997e6214d6` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:45.712475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07716b3a-59f3-4a8e-a834-34d44c22b12e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd88c8e9166940d8a09657997e6214d6)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:45.714392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07716b3a-59f3-4a8e-a834-34d44c22b12e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:45.762280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-07716b3a-59f3-4a8e-a834-34d44c22b12e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd88c8e9166940d8a09657997e6214d6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:45.769531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-07716b3a-59f3-4a8e-a834-34d44c22b12e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd88c8e9166940d8a09657997e6214d6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:45.776370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-07716b3a-59f3-4a8e-a834-34d44c22b12e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd88c8e9166940d8a09657997e6214d6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:45.777047 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-07716b3a-59f3-4a8e-a834-34d44c22b12e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd88c8e9166940d8a09657997e6214d6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:45.777657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-07716b3a-59f3-4a8e-a834-34d44c22b12e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd88c8e9166940d8a09657997e6214d6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:45.795434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1024/2034] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:45 2022] DELETE /identity/v3/users/fd88c8e9166940d8a09657997e6214d6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:45.969340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2dbee-f1d2-481e-888d-2ad61512ff4f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:45.985940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2dbee-f1d2-481e-888d-2ad61512ff4f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:46.099085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:46.099988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.100269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.100542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45 admin admin] PATH_INFO: `/v3/projects/89676dbc35634ec7994040c93f693f7c` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.101726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89676dbc35634ec7994040c93f693f7c)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:46.103807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:46.151834 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89676dbc35634ec7994040c93f693f7c', 'request_id': 'req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:46.157795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89676dbc35634ec7994040c93f693f7c', 'request_id': 'req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:46.165376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89676dbc35634ec7994040c93f693f7c', 'request_id': 'req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:46.175721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45 admin admin] The token cache is being invalidate because project 89676dbc35634ec7994040c93f693f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:46.176005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:46.176618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f8216cf7-403f-46a2-bcc4-7fa8cee29d45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:46.178150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1011/2035] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:45 2022] DELETE /identity/v3/projects/89676dbc35634ec7994040c93f693f7c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:46.183487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff37615-70c2-4234-8e16-cb6ebcab88b6 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.183783 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff37615-70c2-4234-8e16-cb6ebcab88b6 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.184063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff37615-70c2-4234-8e16-cb6ebcab88b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.205737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-dff37615-70c2-4234-8e16-cb6ebcab88b6 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:46.207051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-dff37615-70c2-4234-8e16-cb6ebcab88b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:46.227193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:46.267998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1025/2036] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:46.275863 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a56ffa-ed32-480d-8506-541436bef987 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.276159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a56ffa-ed32-480d-8506-541436bef987 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.276470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a56ffa-ed32-480d-8506-541436bef987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.294891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-94a56ffa-ed32-480d-8506-541436bef987 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:46.296954 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-94a56ffa-ed32-480d-8506-541436bef987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:46.304838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1012/2037] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:46.308875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff37615-70c2-4234-8e16-cb6ebcab88b6 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:46.319054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff37615-70c2-4234-8e16-cb6ebcab88b6 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:46.323627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c87ea6-6ddd-434d-9df2-1e36b1de7396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:46.324264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c87ea6-6ddd-434d-9df2-1e36b1de7396 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.324550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c87ea6-6ddd-434d-9df2-1e36b1de7396 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.324824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c87ea6-6ddd-434d-9df2-1e36b1de7396 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.325547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c87ea6-6ddd-434d-9df2-1e36b1de7396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:46.325853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c87ea6-6ddd-434d-9df2-1e36b1de7396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:46.327559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c87ea6-6ddd-434d-9df2-1e36b1de7396 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:46.339251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1026/2038] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:55:46.343564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a56ffa-ed32-480d-8506-541436bef987 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:46.352568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a56ffa-ed32-480d-8506-541436bef987 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:46.358454 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc3d70-0d98-453b-ae4d-e7c9c77aa661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:46.359224 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc3d70-0d98-453b-ae4d-e7c9c77aa661 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.359505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc3d70-0d98-453b-ae4d-e7c9c77aa661 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.359781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc3d70-0d98-453b-ae4d-e7c9c77aa661 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.360463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc3d70-0d98-453b-ae4d-e7c9c77aa661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:46.362443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc3d70-0d98-453b-ae4d-e7c9c77aa661 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:46.388308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1013/2039] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:46 2022] POST /identity/v3/projects => generated 401 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:46.392450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c87ea6-6ddd-434d-9df2-1e36b1de7396 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:46.402618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c87ea6-6ddd-434d-9df2-1e36b1de7396 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:46.457469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8202aa2-59d7-4fe0-9265-11d7ea984560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:46.458119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8202aa2-59d7-4fe0-9265-11d7ea984560 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.458401 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8202aa2-59d7-4fe0-9265-11d7ea984560 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.458675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8202aa2-59d7-4fe0-9265-11d7ea984560 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.459392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8202aa2-59d7-4fe0-9265-11d7ea984560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:46.461293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8202aa2-59d7-4fe0-9265-11d7ea984560 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:46.481783 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1027/2040] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:46 2022] POST /identity/v3/users => generated 301 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:46.486545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc3d70-0d98-453b-ae4d-e7c9c77aa661 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:46.496813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc3d70-0d98-453b-ae4d-e7c9c77aa661 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:46.502325 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8987322-1883-473a-8b29-c8cf63dd184d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:46.503071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8987322-1883-473a-8b29-c8cf63dd184d admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.503362 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8987322-1883-473a-8b29-c8cf63dd184d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.503903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8987322-1883-473a-8b29-c8cf63dd184d admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.504650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8987322-1883-473a-8b29-c8cf63dd184d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:46.506241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8987322-1883-473a-8b29-c8cf63dd184d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:46.517814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1014/2041] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:46.521784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8202aa2-59d7-4fe0-9265-11d7ea984560 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:46.532674 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8202aa2-59d7-4fe0-9265-11d7ea984560 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:46.537274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e3a9a-efeb-4858-9cf3-a504b7b7759d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:46.537952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e3a9a-efeb-4858-9cf3-a504b7b7759d admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.538238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e3a9a-efeb-4858-9cf3-a504b7b7759d admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.538516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e3a9a-efeb-4858-9cf3-a504b7b7759d admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.539282 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e3a9a-efeb-4858-9cf3-a504b7b7759d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:46.541077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e3a9a-efeb-4858-9cf3-a504b7b7759d admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:46.551188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1028/2042] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:46.555353 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8987322-1883-473a-8b29-c8cf63dd184d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:46.565118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8987322-1883-473a-8b29-c8cf63dd184d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:46.569667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ee0c7-d83b-4d00-a931-75ef826d752f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:46.570532 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ee0c7-d83b-4d00-a931-75ef826d752f admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.570813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ee0c7-d83b-4d00-a931-75ef826d752f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.571123 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ee0c7-d83b-4d00-a931-75ef826d752f admin admin] PATH_INFO: `/v3/projects/3cb61bb43009443a91b06496af8efea1/users/6b480645627e45a7818ff32071e61c34/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.585588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ee0c7-d83b-4d00-a931-75ef826d752f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cb61bb43009443a91b06496af8efea1, user_id=6b480645627e45a7818ff32071e61c34, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:46.588233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ee0c7-d83b-4d00-a931-75ef826d752f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:46.590393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:46.603226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1015/2043] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:46 2022] PUT /identity/v3/projects/3cb61bb43009443a91b06496af8efea1/users/6b480645627e45a7818ff32071e61c34/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:46.614640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3ee51-5766-4c1e-8841-27111ab6c3e6 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.614999 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3ee51-5766-4c1e-8841-27111ab6c3e6 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.615336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3ee51-5766-4c1e-8841-27111ab6c3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.642669 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-acc3ee51-5766-4c1e-8841-27111ab6c3e6 None None] MFA Rules not processed for user `6b480645627e45a7818ff32071e61c34`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:46.643896 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-acc3ee51-5766-4c1e-8841-27111ab6c3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:46.709734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1029/2044] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:46.714470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ee0c7-d83b-4d00-a931-75ef826d752f admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:46.778690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ee0c7-d83b-4d00-a931-75ef826d752f admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:46.783838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc9b3d-e6b0-41be-bdf1-0f5c719e7342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:46.784494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc9b3d-e6b0-41be-bdf1-0f5c719e7342 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.784775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc9b3d-e6b0-41be-bdf1-0f5c719e7342 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.785049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc9b3d-e6b0-41be-bdf1-0f5c719e7342 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.785766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc9b3d-e6b0-41be-bdf1-0f5c719e7342 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:46.787692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc9b3d-e6b0-41be-bdf1-0f5c719e7342 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:46.817030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1016/2045] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:46 2022] POST /identity/v3/projects => generated 401 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:46.818305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3ee51-5766-4c1e-8841-27111ab6c3e6 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:46.829779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3ee51-5766-4c1e-8841-27111ab6c3e6 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:46.848286 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:46.890751 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07517a6-c873-459c-9a11-78d32362f6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:46.891467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07517a6-c873-459c-9a11-78d32362f6a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.891748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07517a6-c873-459c-9a11-78d32362f6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.892028 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07517a6-c873-459c-9a11-78d32362f6a7 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.892738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07517a6-c873-459c-9a11-78d32362f6a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:46.894634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07517a6-c873-459c-9a11-78d32362f6a7 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:46.914761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1030/2046] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:46 2022] POST /identity/v3/users => generated 307 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:46.918705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc9b3d-e6b0-41be-bdf1-0f5c719e7342 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:46.929036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc9b3d-e6b0-41be-bdf1-0f5c719e7342 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:46.933729 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bd277-4b17-4558-8d50-ddaea78b9d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:46.934384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bd277-4b17-4558-8d50-ddaea78b9d82 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.934669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bd277-4b17-4558-8d50-ddaea78b9d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.934980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bd277-4b17-4558-8d50-ddaea78b9d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.936239 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534bd277-4b17-4558-8d50-ddaea78b9d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:46.938102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534bd277-4b17-4558-8d50-ddaea78b9d82 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:46.948509 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1017/2047] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:46.953167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07517a6-c873-459c-9a11-78d32362f6a7 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:46.963104 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07517a6-c873-459c-9a11-78d32362f6a7 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:46.967748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f8834-6322-462e-82d1-7688d6ed4602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:46.968834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f8834-6322-462e-82d1-7688d6ed4602 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:46.969070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f8834-6322-462e-82d1-7688d6ed4602 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:46.969348 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f8834-6322-462e-82d1-7688d6ed4602 admin admin] PATH_INFO: `/v3/projects/0611196bb83343e882ca83200dc647c0/users/d56fba371a7e4e41ae26c70dbcdae4fe/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:46.984300 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f8834-6322-462e-82d1-7688d6ed4602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0611196bb83343e882ca83200dc647c0, user_id=d56fba371a7e4e41ae26c70dbcdae4fe, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:46.986979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f8834-6322-462e-82d1-7688d6ed4602 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:47.000326 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1031/2048] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:46 2022] PUT /identity/v3/projects/0611196bb83343e882ca83200dc647c0/users/d56fba371a7e4e41ae26c70dbcdae4fe/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:47.004447 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bd277-4b17-4558-8d50-ddaea78b9d82 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:47.014478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bd277-4b17-4558-8d50-ddaea78b9d82 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:47.070768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dc1e7-8324-4139-b3d4-1325c51ba323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:47.071484 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dc1e7-8324-4139-b3d4-1325c51ba323 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:47.071772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dc1e7-8324-4139-b3d4-1325c51ba323 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:47.072051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dc1e7-8324-4139-b3d4-1325c51ba323 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:47.072791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dc1e7-8324-4139-b3d4-1325c51ba323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:47.074702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dc1e7-8324-4139-b3d4-1325c51ba323 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:47.087864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1018/2049] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:47.089063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f8834-6322-462e-82d1-7688d6ed4602 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:47.099788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f8834-6322-462e-82d1-7688d6ed4602 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:47.105063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8b2d1-7f9a-4fa2-9650-401146c02490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:47.106068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8b2d1-7f9a-4fa2-9650-401146c02490 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:47.106460 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8b2d1-7f9a-4fa2-9650-401146c02490 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:47.106836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8b2d1-7f9a-4fa2-9650-401146c02490 admin admin] PATH_INFO: `/v3/projects/0611196bb83343e882ca83200dc647c0/users/d56fba371a7e4e41ae26c70dbcdae4fe/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:47.109616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8b2d1-7f9a-4fa2-9650-401146c02490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0611196bb83343e882ca83200dc647c0, user_id=d56fba371a7e4e41ae26c70dbcdae4fe, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:47.112526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8b2d1-7f9a-4fa2-9650-401146c02490 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:47.125518 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1032/2050] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:47 2022] PUT /identity/v3/projects/0611196bb83343e882ca83200dc647c0/users/d56fba371a7e4e41ae26c70dbcdae4fe/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:47.136234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aa2ee-d543-48d9-9ab3-4532bc664e76 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:47.136533 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aa2ee-d543-48d9-9ab3-4532bc664e76 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:47.136820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aa2ee-d543-48d9-9ab3-4532bc664e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:47.167995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-1b1aa2ee-d543-48d9-9ab3-4532bc664e76 None None] MFA Rules not processed for user `d56fba371a7e4e41ae26c70dbcdae4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:47.169127 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1b1aa2ee-d543-48d9-9ab3-4532bc664e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:47.222766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:47.245329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1019/2051] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:47 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:47.256495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8b2d1-7f9a-4fa2-9650-401146c02490 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:47.258239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9cc8b2d1-7f9a-4fa2-9650-401146c02490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:47.315558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8b2d1-7f9a-4fa2-9650-401146c02490 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:47.319337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02492470-6c55-443f-943f-6e5009e3bf9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:47.319970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02492470-6c55-443f-943f-6e5009e3bf9b service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:47.320255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02492470-6c55-443f-943f-6e5009e3bf9b service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:47.320533 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02492470-6c55-443f-943f-6e5009e3bf9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:47.330050 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02492470-6c55-443f-943f-6e5009e3bf9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:47.331511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02492470-6c55-443f-943f-6e5009e3bf9b service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:47.340760 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1033/2052] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:47 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:47.417300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1aa2ee-d543-48d9-9ab3-4532bc664e76 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:47.426918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1aa2ee-d543-48d9-9ab3-4532bc664e76 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:47.430455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd868c-d886-410d-b454-62ffb6171c39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:47.431112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd868c-d886-410d-b454-62ffb6171c39 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:47.431395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd868c-d886-410d-b454-62ffb6171c39 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:47.431668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd868c-d886-410d-b454-62ffb6171c39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:47.490391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd868c-d886-410d-b454-62ffb6171c39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:47.491801 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd868c-d886-410d-b454-62ffb6171c39 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:47.501776 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1020/2053] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:50.518627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02492470-6c55-443f-943f-6e5009e3bf9b service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:50.520148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-02492470-6c55-443f-943f-6e5009e3bf9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:50.628309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02492470-6c55-443f-943f-6e5009e3bf9b service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:50.649480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a10289-1c0c-4a3e-9a01-5fbb59dcd517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:50.650458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a10289-1c0c-4a3e-9a01-5fbb59dcd517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:50.650823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a10289-1c0c-4a3e-9a01-5fbb59dcd517 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:50.651199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a10289-1c0c-4a3e-9a01-5fbb59dcd517 admin admin] PATH_INFO: `/v3/users/8d726f3617b24175a4aa6421e01ecb68` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:50.652930 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a10289-1c0c-4a3e-9a01-5fbb59dcd517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d726f3617b24175a4aa6421e01ecb68)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:50.655865 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a10289-1c0c-4a3e-9a01-5fbb59dcd517 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:50.691127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:50.739496 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-00a10289-1c0c-4a3e-9a01-5fbb59dcd517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d726f3617b24175a4aa6421e01ecb68'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:50.752871 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-00a10289-1c0c-4a3e-9a01-5fbb59dcd517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d726f3617b24175a4aa6421e01ecb68'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:50.767981 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-00a10289-1c0c-4a3e-9a01-5fbb59dcd517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d726f3617b24175a4aa6421e01ecb68'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:50.771403 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd868c-d886-410d-b454-62ffb6171c39 service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:50.772970 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-8dfd868c-d886-410d-b454-62ffb6171c39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:50.776032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-00a10289-1c0c-4a3e-9a01-5fbb59dcd517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d726f3617b24175a4aa6421e01ecb68'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:50.778219 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-00a10289-1c0c-4a3e-9a01-5fbb59dcd517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d726f3617b24175a4aa6421e01ecb68'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:50.792396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd868c-d886-410d-b454-62ffb6171c39 service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:50.803423 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9936ba7-7aea-4f03-906a-1395eb056b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:50.804684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9936ba7-7aea-4f03-906a-1395eb056b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:50.805798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9936ba7-7aea-4f03-906a-1395eb056b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:50.806398 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9936ba7-7aea-4f03-906a-1395eb056b79 admin admin] PATH_INFO: `/v3/users/8eb7ad081a324ea78983ec0674578d08` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:50.808001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9936ba7-7aea-4f03-906a-1395eb056b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb7ad081a324ea78983ec0674578d08)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:50.811745 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9936ba7-7aea-4f03-906a-1395eb056b79 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:50.819983 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1034/2054] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:50 2022] DELETE /identity/v3/users/8d726f3617b24175a4aa6421e01ecb68 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:50.822053 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29df38b-8b5b-4ab3-a94a-1a4c5508ffb2 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:50.822351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29df38b-8b5b-4ab3-a94a-1a4c5508ffb2 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:50.822631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29df38b-8b5b-4ab3-a94a-1a4c5508ffb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:50.858622 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e29df38b-8b5b-4ab3-a94a-1a4c5508ffb2 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:50.860083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e29df38b-8b5b-4ab3-a94a-1a4c5508ffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:50.900564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f9936ba7-7aea-4f03-906a-1395eb056b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb7ad081a324ea78983ec0674578d08'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:50.916507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f9936ba7-7aea-4f03-906a-1395eb056b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb7ad081a324ea78983ec0674578d08'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:50.919229 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f9936ba7-7aea-4f03-906a-1395eb056b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb7ad081a324ea78983ec0674578d08'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:50.940483 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f9936ba7-7aea-4f03-906a-1395eb056b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb7ad081a324ea78983ec0674578d08'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:50.954811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1035/2055] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:50.969517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f9936ba7-7aea-4f03-906a-1395eb056b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb7ad081a324ea78983ec0674578d08'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:50.970574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29df38b-8b5b-4ab3-a94a-1a4c5508ffb2 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:50.972897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e29df38b-8b5b-4ab3-a94a-1a4c5508ffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:50.976785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1021/2056] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:50 2022] DELETE /identity/v3/users/8eb7ad081a324ea78983ec0674578d08 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:50.981443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8993856-65e0-4241-b517-1324451e5df3 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:50.982067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8993856-65e0-4241-b517-1324451e5df3 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:50.982452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8993856-65e0-4241-b517-1324451e5df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:51.016402 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a8993856-65e0-4241-b517-1324451e5df3 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:51.019102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a8993856-65e0-4241-b517-1324451e5df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:51.079441 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29df38b-8b5b-4ab3-a94a-1a4c5508ffb2 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:51.130782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:51.150829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1022/2057] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:51.159745 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8993856-65e0-4241-b517-1324451e5df3 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:51.162369 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a8993856-65e0-4241-b517-1324451e5df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:51.176061 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c63616-9a9c-4645-935f-3a8ee88aaef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:51.177054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c63616-9a9c-4645-935f-3a8ee88aaef0 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:51.177445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c63616-9a9c-4645-935f-3a8ee88aaef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:51.178299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c63616-9a9c-4645-935f-3a8ee88aaef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:51.181583 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-17c63616-9a9c-4645-935f-3a8ee88aaef0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:51.207497 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:51.210420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c63616-9a9c-4645-935f-3a8ee88aaef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:51.213807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c63616-9a9c-4645-935f-3a8ee88aaef0 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:51.230501 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1036/2058] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:51.270056 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8993856-65e0-4241-b517-1324451e5df3 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:51.273830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d07609-be72-4cee-89d2-10300aa90b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:51.274484 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d07609-be72-4cee-89d2-10300aa90b0b service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:51.274765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d07609-be72-4cee-89d2-10300aa90b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:51.275088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d07609-be72-4cee-89d2-10300aa90b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:51.284797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d07609-be72-4cee-89d2-10300aa90b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:51.286220 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d07609-be72-4cee-89d2-10300aa90b0b service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:51.297332 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1023/2059] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:51.598771 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c63616-9a9c-4645-935f-3a8ee88aaef0 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:51.600248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-17c63616-9a9c-4645-935f-3a8ee88aaef0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:51.612284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c63616-9a9c-4645-935f-3a8ee88aaef0 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:51.619921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:51.620984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:51.621377 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:51.621840 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] PATH_INFO: `/v3/users/6b480645627e45a7818ff32071e61c34` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:51.623369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b480645627e45a7818ff32071e61c34)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:51.626006 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:51.697023 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b480645627e45a7818ff32071e61c34'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:51.711323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b480645627e45a7818ff32071e61c34'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:51.723866 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b480645627e45a7818ff32071e61c34'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:51.725661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b480645627e45a7818ff32071e61c34'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:51.726229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b480645627e45a7818ff32071e61c34'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:51.746969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1037/2060] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:51 2022] DELETE /identity/v3/users/6b480645627e45a7818ff32071e61c34 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:51.750915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d1912-64a4-4a9a-9552-f905e49a29ab None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:51.751213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d1912-64a4-4a9a-9552-f905e49a29ab None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:51.751501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d1912-64a4-4a9a-9552-f905e49a29ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:51.775896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-9d2d1912-64a4-4a9a-9552-f905e49a29ab None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:51.777647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9d2d1912-64a4-4a9a-9552-f905e49a29ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:51.789667 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:51.791609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:51.875620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1024/2061] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:51.887605 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d1912-64a4-4a9a-9552-f905e49a29ab None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:51.889667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9d2d1912-64a4-4a9a-9552-f905e49a29ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:51.905073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0beef-b734-4106-9421-60908f324c7d admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:51.912557 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:51.913629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:51.914622 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:51.915109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] PATH_INFO: `/v3/projects/1e864348a7104604a3f73fe949ed8951` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:51.918156 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e864348a7104604a3f73fe949ed8951)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:51.920772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:51.965659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d1912-64a4-4a9a-9552-f905e49a29ab None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:51.971542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e327b7-087b-4078-8880-56badc3fe56a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:51.972209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e327b7-087b-4078-8880-56badc3fe56a service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:51.972496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e327b7-087b-4078-8880-56badc3fe56a service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:51.972965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e327b7-087b-4078-8880-56badc3fe56a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:51.984640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e327b7-087b-4078-8880-56badc3fe56a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:51.985034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e864348a7104604a3f73fe949ed8951', 'request_id': 'req-620e77bf-8bef-4907-b89d-fdf9adf1023c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:51.995521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e327b7-087b-4078-8880-56badc3fe56a service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:52.011847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e864348a7104604a3f73fe949ed8951', 'request_id': 'req-620e77bf-8bef-4907-b89d-fdf9adf1023c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.012236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1025/2062] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:55:52.012236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e327b7-087b-4078-8880-56badc3fe56a service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:52.012972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-71e327b7-087b-4078-8880-56badc3fe56a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:52.024918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e864348a7104604a3f73fe949ed8951', 'request_id': 'req-620e77bf-8bef-4907-b89d-fdf9adf1023c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.030566 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e327b7-087b-4078-8880-56badc3fe56a service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:52.036629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:52.041917 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] The token cache is being invalidate because project 1e864348a7104604a3f73fe949ed8951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:52.042348 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.043353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.048332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1038/2063] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:51 2022] DELETE /identity/v3/projects/1e864348a7104604a3f73fe949ed8951 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:52.052069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:52.056681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:52.087696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:52.115890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:52.118554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:52.119150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:52.120018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] PATH_INFO: `/v3/projects/12e0c074192d45cc8056a5cbb526932b` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:52.123875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] RBAC: Authorizing `identity:delete_project(project_id=12e0c074192d45cc8056a5cbb526932b)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:52.128234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:52.166736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e77bf-8bef-4907-b89d-fdf9adf1023c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:52.172324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:52.173219 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:52.173505 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:52.173782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] PATH_INFO: `/v3/users/f16f60ec9ee9421ca81a3c4a6e089273` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:52.175103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f16f60ec9ee9421ca81a3c4a6e089273)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:52.177214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:52.254000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12e0c074192d45cc8056a5cbb526932b', 'request_id': 'req-9310f813-6ce7-46ce-be29-48bdf85cbafb'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.273136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f16f60ec9ee9421ca81a3c4a6e089273'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.283191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f16f60ec9ee9421ca81a3c4a6e089273'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.283999 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12e0c074192d45cc8056a5cbb526932b', 'request_id': 'req-9310f813-6ce7-46ce-be29-48bdf85cbafb'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.297516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f16f60ec9ee9421ca81a3c4a6e089273'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.298544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f16f60ec9ee9421ca81a3c4a6e089273'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.299360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f16f60ec9ee9421ca81a3c4a6e089273'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.311755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12e0c074192d45cc8056a5cbb526932b', 'request_id': 'req-9310f813-6ce7-46ce-be29-48bdf85cbafb'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.336952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1039/2064] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:52 2022] DELETE /identity/v3/users/f16f60ec9ee9421ca81a3c4a6e089273 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:52.342893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] The token cache is being invalidate because project 12e0c074192d45cc8056a5cbb526932b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:52.343167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.347835 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.350173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1026/2065] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:52 2022] DELETE /identity/v3/projects/12e0c074192d45cc8056a5cbb526932b => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:52.353982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:52.355914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:52.407949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:52.410238 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:52.471585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740cdf8-abf1-43a8-8489-c6c19c330574 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:52.479932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:52.481813 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:52.482112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:52.482275 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] PATH_INFO: `/v3/users/10c7699a7e444af38f448f16d2bce038` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:52.484080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10c7699a7e444af38f448f16d2bce038)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:52.486502 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:52.520388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:52.532589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310f813-6ce7-46ce-be29-48bdf85cbafb admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:52.540342 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:52.541639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:52.542134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:52.542621 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] PATH_INFO: `/v3/projects/3cb61bb43009443a91b06496af8efea1` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:52.544723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cb61bb43009443a91b06496af8efea1)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:52.548858 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:52.572146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10c7699a7e444af38f448f16d2bce038'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.587806 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10c7699a7e444af38f448f16d2bce038'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.602393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10c7699a7e444af38f448f16d2bce038'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.603693 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10c7699a7e444af38f448f16d2bce038'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.604533 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10c7699a7e444af38f448f16d2bce038'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.649330 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cb61bb43009443a91b06496af8efea1', 'request_id': 'req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.661845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cb61bb43009443a91b06496af8efea1', 'request_id': 'req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.666150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1040/2066] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:52 2022] DELETE /identity/v3/users/10c7699a7e444af38f448f16d2bce038 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:52.689478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cb61bb43009443a91b06496af8efea1', 'request_id': 'req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.726760 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] The token cache is being invalidate because project 3cb61bb43009443a91b06496af8efea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:52.727086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.731819 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:52.734401 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1027/2067] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:52 2022] DELETE /identity/v3/projects/3cb61bb43009443a91b06496af8efea1 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:52.739443 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:52.743680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:52.934212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d1df6-1fb7-46fe-a213-3367102fe9a2 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:52.942046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:52.944232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:52.944537 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:52.944793 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] PATH_INFO: `/v3/users/d56fba371a7e4e41ae26c70dbcdae4fe` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:52.946201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d56fba371a7e4e41ae26c70dbcdae4fe)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:52.948870 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:53.036764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd56fba371a7e4e41ae26c70dbcdae4fe'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.059022 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd56fba371a7e4e41ae26c70dbcdae4fe'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.061508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:53.063593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:53.075154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:53.081412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd56fba371a7e4e41ae26c70dbcdae4fe'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.083260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd56fba371a7e4e41ae26c70dbcdae4fe'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.084613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd56fba371a7e4e41ae26c70dbcdae4fe'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.137310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1041/2068] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:52 2022] DELETE /identity/v3/users/d56fba371a7e4e41ae26c70dbcdae4fe => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:53.188567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ce979-ccb8-469b-a9f6-73ccc1f79f37 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:53.258183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:53.260192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:53.282794 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609c71b-18cb-478a-9198-0dd2bb0b2a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:53.283725 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609c71b-18cb-478a-9198-0dd2bb0b2a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:53.284589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609c71b-18cb-478a-9198-0dd2bb0b2a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:53.284877 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609c71b-18cb-478a-9198-0dd2bb0b2a13 admin admin] PATH_INFO: `/v3/projects/ce1fa9ab40324ea0b6a888cb64e28bcf` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:53.286220 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609c71b-18cb-478a-9198-0dd2bb0b2a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce1fa9ab40324ea0b6a888cb64e28bcf)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:53.290646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:53.298907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609c71b-18cb-478a-9198-0dd2bb0b2a13 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:53.365483 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a609c71b-18cb-478a-9198-0dd2bb0b2a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce1fa9ab40324ea0b6a888cb64e28bcf', 'request_id': 'req-a609c71b-18cb-478a-9198-0dd2bb0b2a13'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.366154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa21a48-d957-481a-8543-d275bd4ee17c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:53.374300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a609c71b-18cb-478a-9198-0dd2bb0b2a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce1fa9ab40324ea0b6a888cb64e28bcf', 'request_id': 'req-a609c71b-18cb-478a-9198-0dd2bb0b2a13'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.386181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a609c71b-18cb-478a-9198-0dd2bb0b2a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce1fa9ab40324ea0b6a888cb64e28bcf', 'request_id': 'req-a609c71b-18cb-478a-9198-0dd2bb0b2a13'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.422284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a609c71b-18cb-478a-9198-0dd2bb0b2a13 admin admin] The token cache is being invalidate because project ce1fa9ab40324ea0b6a888cb64e28bcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:53.422589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a609c71b-18cb-478a-9198-0dd2bb0b2a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.425244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-a609c71b-18cb-478a-9198-0dd2bb0b2a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.432063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1028/2069] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:53 2022] DELETE /identity/v3/projects/ce1fa9ab40324ea0b6a888cb64e28bcf => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:53.436859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf96a6-2a5e-469e-96a5-cfe83a33f925 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:53.437212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf96a6-2a5e-469e-96a5-cfe83a33f925 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:53.437503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf96a6-2a5e-469e-96a5-cfe83a33f925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:53.448521 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177cbb7-eeb1-4b1a-904b-21701d913c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:53.449856 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177cbb7-eeb1-4b1a-904b-21701d913c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:53.450332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177cbb7-eeb1-4b1a-904b-21701d913c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:53.452296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177cbb7-eeb1-4b1a-904b-21701d913c06 admin admin] PATH_INFO: `/v3/projects/ca436d7ffa234f88ae0a594b878ae260` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:53.455163 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177cbb7-eeb1-4b1a-904b-21701d913c06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca436d7ffa234f88ae0a594b878ae260)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:53.458011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177cbb7-eeb1-4b1a-904b-21701d913c06 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:53.463808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-eacf96a6-2a5e-469e-96a5-cfe83a33f925 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:53.465182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-eacf96a6-2a5e-469e-96a5-cfe83a33f925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:53.480608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1029/2070] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:53.489503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f1449-3f1f-483d-b220-2c25c5e05c62 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:53.490223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f1449-3f1f-483d-b220-2c25c5e05c62 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:53.490390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f1449-3f1f-483d-b220-2c25c5e05c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:53.517471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-7b5f1449-3f1f-483d-b220-2c25c5e05c62 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:53.518774 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-7b5f1449-3f1f-483d-b220-2c25c5e05c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:53.533562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d177cbb7-eeb1-4b1a-904b-21701d913c06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca436d7ffa234f88ae0a594b878ae260', 'request_id': 'req-d177cbb7-eeb1-4b1a-904b-21701d913c06'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.540938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d177cbb7-eeb1-4b1a-904b-21701d913c06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca436d7ffa234f88ae0a594b878ae260', 'request_id': 'req-d177cbb7-eeb1-4b1a-904b-21701d913c06'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.556244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d177cbb7-eeb1-4b1a-904b-21701d913c06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca436d7ffa234f88ae0a594b878ae260', 'request_id': 'req-d177cbb7-eeb1-4b1a-904b-21701d913c06'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.564902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:53.574011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d177cbb7-eeb1-4b1a-904b-21701d913c06 admin admin] The token cache is being invalidate because project ca436d7ffa234f88ae0a594b878ae260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:53.574371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d177cbb7-eeb1-4b1a-904b-21701d913c06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.575595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d177cbb7-eeb1-4b1a-904b-21701d913c06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.577797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1042/2071] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:53 2022] DELETE /identity/v3/projects/ca436d7ffa234f88ae0a594b878ae260 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:53.583092 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b4d60-9c9b-47c9-919f-55c761f25972 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:53.583920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b4d60-9c9b-47c9-919f-55c761f25972 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:53.584407 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b4d60-9c9b-47c9-919f-55c761f25972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:53.614001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-d28b4d60-9c9b-47c9-919f-55c761f25972 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:53.616117 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d28b4d60-9c9b-47c9-919f-55c761f25972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:53.624172 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1030/2072] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:55:53.625297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f1449-3f1f-483d-b220-2c25c5e05c62 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:53.627017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-7b5f1449-3f1f-483d-b220-2c25c5e05c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:53.635293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1043/2073] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:53.636495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b4d60-9c9b-47c9-919f-55c761f25972 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:53.648111 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f1449-3f1f-483d-b220-2c25c5e05c62 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:53.650903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b4d60-9c9b-47c9-919f-55c761f25972 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:53.653021 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:53.653937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:53.654218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:53.655134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] PATH_INFO: `/v3/projects/0611196bb83343e882ca83200dc647c0` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:53.656579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0611196bb83343e882ca83200dc647c0)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:53.657641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299eda7-044e-4ebf-b613-5aa06e69286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:53.658485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299eda7-044e-4ebf-b613-5aa06e69286f admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:53.658586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:53.658894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299eda7-044e-4ebf-b613-5aa06e69286f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:53.659240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299eda7-044e-4ebf-b613-5aa06e69286f admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:53.660146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299eda7-044e-4ebf-b613-5aa06e69286f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:53.660362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299eda7-044e-4ebf-b613-5aa06e69286f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:53.661977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299eda7-044e-4ebf-b613-5aa06e69286f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:53.676980 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1044/2074] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:55:53.678375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0cba5-f07f-4027-9ca9-f23d4a09413c None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:53.678675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0cba5-f07f-4027-9ca9-f23d4a09413c None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:53.678983 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0cba5-f07f-4027-9ca9-f23d4a09413c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:53.711692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-2af0cba5-f07f-4027-9ca9-f23d4a09413c None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:53.713532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2af0cba5-f07f-4027-9ca9-f23d4a09413c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:53.720497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0611196bb83343e882ca83200dc647c0', 'request_id': 'req-d231b281-3a84-49ef-a4b1-c7c9dc18e592'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.728273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0611196bb83343e882ca83200dc647c0', 'request_id': 'req-d231b281-3a84-49ef-a4b1-c7c9dc18e592'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.737214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0611196bb83343e882ca83200dc647c0', 'request_id': 'req-d231b281-3a84-49ef-a4b1-c7c9dc18e592'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.749673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] The token cache is being invalidate because project 0611196bb83343e882ca83200dc647c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:53.749964 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.750954 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:53.756514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1031/2075] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:53 2022] DELETE /identity/v3/projects/0611196bb83343e882ca83200dc647c0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:53.756691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:53.756867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:53.756867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:53.800070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1045/2076] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:53.803423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e156ac-527d-4b38-9010-18295bbb9b6a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:53.803847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e156ac-527d-4b38-9010-18295bbb9b6a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:53.804135 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e156ac-527d-4b38-9010-18295bbb9b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:53.825005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-01e156ac-527d-4b38-9010-18295bbb9b6a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:53.826723 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-01e156ac-527d-4b38-9010-18295bbb9b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:53.830145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231b281-3a84-49ef-a4b1-c7c9dc18e592 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:53.834808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406fb53-26b6-4394-a2c7-fd73a55802e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:53.835431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1046/2077] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:53.841415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406fb53-26b6-4394-a2c7-fd73a55802e9 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:53.841415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406fb53-26b6-4394-a2c7-fd73a55802e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:53.841415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406fb53-26b6-4394-a2c7-fd73a55802e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:53.841415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406fb53-26b6-4394-a2c7-fd73a55802e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:53.841777 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e156ac-527d-4b38-9010-18295bbb9b6a None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:53.842646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406fb53-26b6-4394-a2c7-fd73a55802e9 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:53.852674 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e156ac-527d-4b38-9010-18295bbb9b6a None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:53.861766 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe049e3-0ebf-4b94-a286-4d7a40fb78cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:53.863014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe049e3-0ebf-4b94-a286-4d7a40fb78cc admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:53.863483 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe049e3-0ebf-4b94-a286-4d7a40fb78cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:53.863940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe049e3-0ebf-4b94-a286-4d7a40fb78cc admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:53.865573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe049e3-0ebf-4b94-a286-4d7a40fb78cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:53.866220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe049e3-0ebf-4b94-a286-4d7a40fb78cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:53.869277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe049e3-0ebf-4b94-a286-4d7a40fb78cc admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:53.872575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1032/2078] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:53 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:53.873961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c0aeb-a4fc-4273-ae2b-e451eb5c6247 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:53.874334 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c0aeb-a4fc-4273-ae2b-e451eb5c6247 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:53.874680 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c0aeb-a4fc-4273-ae2b-e451eb5c6247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:53.893145 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1047/2079] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:55:53.893145 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe049e3-0ebf-4b94-a286-4d7a40fb78cc admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:53.896212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a83c0aeb-a4fc-4273-ae2b-e451eb5c6247 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:53.897500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a83c0aeb-a4fc-4273-ae2b-e451eb5c6247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:53.905476 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe049e3-0ebf-4b94-a286-4d7a40fb78cc admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:53.951144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:53.981577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1033/2080] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:53.985514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c0aeb-a4fc-4273-ae2b-e451eb5c6247 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:53.996345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c0aeb-a4fc-4273-ae2b-e451eb5c6247 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.004342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865bb06-2d4f-4e0c-9e8a-49fb85863684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.004614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f446d33-052e-4b96-99bb-95d4fcbcca11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.004614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f446d33-052e-4b96-99bb-95d4fcbcca11 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.004614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f446d33-052e-4b96-99bb-95d4fcbcca11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.004614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f446d33-052e-4b96-99bb-95d4fcbcca11 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.004885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865bb06-2d4f-4e0c-9e8a-49fb85863684 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.004885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865bb06-2d4f-4e0c-9e8a-49fb85863684 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.005027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f446d33-052e-4b96-99bb-95d4fcbcca11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.005098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865bb06-2d4f-4e0c-9e8a-49fb85863684 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.005464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865bb06-2d4f-4e0c-9e8a-49fb85863684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.007808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865bb06-2d4f-4e0c-9e8a-49fb85863684 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.013204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f446d33-052e-4b96-99bb-95d4fcbcca11 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.033135 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1048/2081] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:53 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:54.034234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865bb06-2d4f-4e0c-9e8a-49fb85863684 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.046244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865bb06-2d4f-4e0c-9e8a-49fb85863684 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.050892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1034/2082] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:53 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:54.051141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71083c2c-6adf-4a50-802c-9395a4fa4cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.051789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f446d33-052e-4b96-99bb-95d4fcbcca11 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.051895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71083c2c-6adf-4a50-802c-9395a4fa4cfe admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.052133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71083c2c-6adf-4a50-802c-9395a4fa4cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.052407 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71083c2c-6adf-4a50-802c-9395a4fa4cfe admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.053143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71083c2c-6adf-4a50-802c-9395a4fa4cfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:54.053533 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71083c2c-6adf-4a50-802c-9395a4fa4cfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.055165 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71083c2c-6adf-4a50-802c-9395a4fa4cfe admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.084746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1049/2083] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:55:54.089423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71083c2c-6adf-4a50-802c-9395a4fa4cfe admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.103448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71083c2c-6adf-4a50-802c-9395a4fa4cfe admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.124765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f446d33-052e-4b96-99bb-95d4fcbcca11 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.131070 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ba969-274e-4bd7-ab18-d157520808d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.131744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ba969-274e-4bd7-ab18-d157520808d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.132064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ba969-274e-4bd7-ab18-d157520808d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.132344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ba969-274e-4bd7-ab18-d157520808d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.133035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ba969-274e-4bd7-ab18-d157520808d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.134707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ba969-274e-4bd7-ab18-d157520808d5 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.148073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1035/2084] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:54.148912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ba969-274e-4bd7-ab18-d157520808d5 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.163100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ba969-274e-4bd7-ab18-d157520808d5 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.168929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9963091-6555-4df1-ab98-1f0408dd972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.169844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9963091-6555-4df1-ab98-1f0408dd972c admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.170898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9963091-6555-4df1-ab98-1f0408dd972c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.171600 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9963091-6555-4df1-ab98-1f0408dd972c admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.172684 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9963091-6555-4df1-ab98-1f0408dd972c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.175772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9963091-6555-4df1-ab98-1f0408dd972c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.200877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5980d02-aa65-4407-b83e-2b2fc4a20e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.201723 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5980d02-aa65-4407-b83e-2b2fc4a20e79 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.202083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5980d02-aa65-4407-b83e-2b2fc4a20e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.202655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5980d02-aa65-4407-b83e-2b2fc4a20e79 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.203459 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5980d02-aa65-4407-b83e-2b2fc4a20e79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.205409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5980d02-aa65-4407-b83e-2b2fc4a20e79 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.207948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1036/2085] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:54 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:54.211914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9963091-6555-4df1-ab98-1f0408dd972c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.227230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1050/2086] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:54 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:54.228028 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5980d02-aa65-4407-b83e-2b2fc4a20e79 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.238746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5980d02-aa65-4407-b83e-2b2fc4a20e79 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.256529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:54.288797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9963091-6555-4df1-ab98-1f0408dd972c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.293639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384303b-45f3-4dcc-820a-03f9041d85db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.294306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384303b-45f3-4dcc-820a-03f9041d85db admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.294599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384303b-45f3-4dcc-820a-03f9041d85db admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.294891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384303b-45f3-4dcc-820a-03f9041d85db admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.296315 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384303b-45f3-4dcc-820a-03f9041d85db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.298310 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384303b-45f3-4dcc-820a-03f9041d85db admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.305084 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964155df-6570-4365-bc71-bdc29ffe96da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.305800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964155df-6570-4365-bc71-bdc29ffe96da admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.306096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964155df-6570-4365-bc71-bdc29ffe96da admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.306389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964155df-6570-4365-bc71-bdc29ffe96da admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.307039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964155df-6570-4365-bc71-bdc29ffe96da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.312079 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964155df-6570-4365-bc71-bdc29ffe96da admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.312260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1037/2087] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:54.312260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384303b-45f3-4dcc-820a-03f9041d85db admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.320007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384303b-45f3-4dcc-820a-03f9041d85db admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.325702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c13d8d-dc9d-4555-a290-ad0ebb116cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.326483 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c13d8d-dc9d-4555-a290-ad0ebb116cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.326969 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c13d8d-dc9d-4555-a290-ad0ebb116cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.327360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c13d8d-dc9d-4555-a290-ad0ebb116cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.328110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c13d8d-dc9d-4555-a290-ad0ebb116cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.329796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c13d8d-dc9d-4555-a290-ad0ebb116cb4 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.332811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1051/2088] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:54 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:54.333599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964155df-6570-4365-bc71-bdc29ffe96da admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.344237 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1038/2089] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:54.345091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c13d8d-dc9d-4555-a290-ad0ebb116cb4 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.347604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964155df-6570-4365-bc71-bdc29ffe96da admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.352624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51c386-8da4-4bba-b36d-61d242f7ac67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.353519 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51c386-8da4-4bba-b36d-61d242f7ac67 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.353809 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51c386-8da4-4bba-b36d-61d242f7ac67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.354089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51c386-8da4-4bba-b36d-61d242f7ac67 admin admin] PATH_INFO: `/v3/projects/35444f2dc65a40828956a113763d3259/users/1b630eac3ac2443d8cb54a7901ca88e7/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.355835 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c13d8d-dc9d-4555-a290-ad0ebb116cb4 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.362072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1bdd9-fd0c-482b-a7f5-35b2a3daa548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.362736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1bdd9-fd0c-482b-a7f5-35b2a3daa548 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.363066 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1bdd9-fd0c-482b-a7f5-35b2a3daa548 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.363347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1bdd9-fd0c-482b-a7f5-35b2a3daa548 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.364078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1bdd9-fd0c-482b-a7f5-35b2a3daa548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.365687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1bdd9-fd0c-482b-a7f5-35b2a3daa548 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.374472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51c386-8da4-4bba-b36d-61d242f7ac67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35444f2dc65a40828956a113763d3259, user_id=1b630eac3ac2443d8cb54a7901ca88e7, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.378028 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51c386-8da4-4bba-b36d-61d242f7ac67 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.379700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1039/2090] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:54.380784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1bdd9-fd0c-482b-a7f5-35b2a3daa548 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.394316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1bdd9-fd0c-482b-a7f5-35b2a3daa548 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.396935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1052/2091] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:54 2022] PUT /identity/v3/projects/35444f2dc65a40828956a113763d3259/users/1b630eac3ac2443d8cb54a7901ca88e7/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:54.397857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51c386-8da4-4bba-b36d-61d242f7ac67 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.417093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:54.475540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51c386-8da4-4bba-b36d-61d242f7ac67 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.477149 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab767672-1d45-4d79-a714-e6b7378cb89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.477843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab767672-1d45-4d79-a714-e6b7378cb89d admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.478238 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab767672-1d45-4d79-a714-e6b7378cb89d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.478518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab767672-1d45-4d79-a714-e6b7378cb89d admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.479273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab767672-1d45-4d79-a714-e6b7378cb89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.480233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3056f-98a6-47eb-809c-b055fe2351f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.480888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3056f-98a6-47eb-809c-b055fe2351f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.481003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab767672-1d45-4d79-a714-e6b7378cb89d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.481177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3056f-98a6-47eb-809c-b055fe2351f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.481454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3056f-98a6-47eb-809c-b055fe2351f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.482180 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3056f-98a6-47eb-809c-b055fe2351f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.484463 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3056f-98a6-47eb-809c-b055fe2351f8 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.492338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1040/2092] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:54.495588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1053/2093] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:54.496538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95ed27-7c3a-49a3-af60-9821c6261adf None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.496838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95ed27-7c3a-49a3-af60-9821c6261adf None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.497121 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95ed27-7c3a-49a3-af60-9821c6261adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.500150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3056f-98a6-47eb-809c-b055fe2351f8 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.510630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3056f-98a6-47eb-809c-b055fe2351f8 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.515943 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d715a3-8c56-4d37-9526-bb05ba6482f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.516845 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d715a3-8c56-4d37-9526-bb05ba6482f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.517095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d715a3-8c56-4d37-9526-bb05ba6482f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.517371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d715a3-8c56-4d37-9526-bb05ba6482f7 admin admin] PATH_INFO: `/v3/projects/d698f3c70e30495c8c3ab15afb488784/users/028c23493a524f14a5996719ad1b9448/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.526388 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-ac95ed27-7c3a-49a3-af60-9821c6261adf None None] MFA Rules not processed for user `1b630eac3ac2443d8cb54a7901ca88e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:54.527699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ac95ed27-7c3a-49a3-af60-9821c6261adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:54.532906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d715a3-8c56-4d37-9526-bb05ba6482f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d698f3c70e30495c8c3ab15afb488784, user_id=028c23493a524f14a5996719ad1b9448, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.535631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d715a3-8c56-4d37-9526-bb05ba6482f7 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.549473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1054/2094] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:54 2022] PUT /identity/v3/projects/d698f3c70e30495c8c3ab15afb488784/users/028c23493a524f14a5996719ad1b9448/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:54.550472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d715a3-8c56-4d37-9526-bb05ba6482f7 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.604786 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1041/2095] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:54.606402 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b221db-f8a7-49f6-98f8-abbf6967459b None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.606780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b221db-f8a7-49f6-98f8-abbf6967459b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.607130 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b221db-f8a7-49f6-98f8-abbf6967459b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.612184 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:54.619123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d715a3-8c56-4d37-9526-bb05ba6482f7 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.625457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bb077-5294-4965-9025-b533a6688619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.626457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bb077-5294-4965-9025-b533a6688619 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.626885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bb077-5294-4965-9025-b533a6688619 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.627177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bb077-5294-4965-9025-b533a6688619 admin admin] PATH_INFO: `/v3/projects/763e30960daa4e638606477f656ae74c/users/22b56ce385f240bfa9ce353747af4ae6/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.635181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-17b221db-f8a7-49f6-98f8-abbf6967459b None None] MFA Rules not processed for user `028c23493a524f14a5996719ad1b9448`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:54.636384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-17b221db-f8a7-49f6-98f8-abbf6967459b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:54.642392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bb077-5294-4965-9025-b533a6688619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=763e30960daa4e638606477f656ae74c, user_id=22b56ce385f240bfa9ce353747af4ae6, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.645060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bb077-5294-4965-9025-b533a6688619 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.657994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1055/2096] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:54 2022] PUT /identity/v3/projects/763e30960daa4e638606477f656ae74c/users/22b56ce385f240bfa9ce353747af4ae6/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:54.663759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bb077-5294-4965-9025-b533a6688619 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.665397 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-679bb077-5294-4965-9025-b533a6688619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:54.708958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1042/2097] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:54.712253 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c1459-b4d4-4e0c-91e2-d38c988224bb None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.712722 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c1459-b4d4-4e0c-91e2-d38c988224bb None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.713169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c1459-b4d4-4e0c-91e2-d38c988224bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.732924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bb077-5294-4965-9025-b533a6688619 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.737949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e276f4-390f-4bc3-9ac6-6d6031e26321 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '666ade4ce1fe447487998d086779389b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.738717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e276f4-390f-4bc3-9ac6-6d6031e26321 service glance] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.739090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e276f4-390f-4bc3-9ac6-6d6031e26321 service glance] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.739872 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e276f4-390f-4bc3-9ac6-6d6031e26321 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.742536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-dd7c1459-b4d4-4e0c-91e2-d38c988224bb None None] MFA Rules not processed for user `22b56ce385f240bfa9ce353747af4ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:54.743881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-dd7c1459-b4d4-4e0c-91e2-d38c988224bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:54.765375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:54.807530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e276f4-390f-4bc3-9ac6-6d6031e26321 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.809193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e276f4-390f-4bc3-9ac6-6d6031e26321 service glance] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.821592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1056/2098] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:54.821770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1043/2099] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:54.822501 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e276f4-390f-4bc3-9ac6-6d6031e26321 service glance] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.825867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c1459-b4d4-4e0c-91e2-d38c988224bb None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.900132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e276f4-390f-4bc3-9ac6-6d6031e26321 service glance] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.905376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5b42b-8043-4cfb-a1f5-690d3777545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.906041 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5b42b-8043-4cfb-a1f5-690d3777545c admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.906323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5b42b-8043-4cfb-a1f5-690d3777545c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.906608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5b42b-8043-4cfb-a1f5-690d3777545c admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.907409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5b42b-8043-4cfb-a1f5-690d3777545c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.909405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5b42b-8043-4cfb-a1f5-690d3777545c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.915615 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c1459-b4d4-4e0c-91e2-d38c988224bb None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:54.921757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffad1a7-6ea7-448d-9918-37112af054b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:54.922319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffad1a7-6ea7-448d-9918-37112af054b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:54.922603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffad1a7-6ea7-448d-9918-37112af054b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:54.922892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffad1a7-6ea7-448d-9918-37112af054b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:54.923588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffad1a7-6ea7-448d-9918-37112af054b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:54.925465 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffad1a7-6ea7-448d-9918-37112af054b1 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:54.939624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1057/2100] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:54 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:54.941050 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5b42b-8043-4cfb-a1f5-690d3777545c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.957221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1044/2101] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:54 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:54.958188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffad1a7-6ea7-448d-9918-37112af054b1 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:54.964829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:55.014688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5b42b-8043-4cfb-a1f5-690d3777545c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:55.034083 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffad1a7-6ea7-448d-9918-37112af054b1 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:55.061717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a32db-ecdf-4cf2-8db7-575327270575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:55.063609 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a32db-ecdf-4cf2-8db7-575327270575 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.065828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a32db-ecdf-4cf2-8db7-575327270575 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.066190 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a32db-ecdf-4cf2-8db7-575327270575 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.066999 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a32db-ecdf-4cf2-8db7-575327270575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:55.070375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a32db-ecdf-4cf2-8db7-575327270575 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:55.114932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1045/2102] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:54 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:55.117471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a32db-ecdf-4cf2-8db7-575327270575 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:55.117656 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee2e32-b381-4251-9038-ac14fabbf3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:55.118412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee2e32-b381-4251-9038-ac14fabbf3b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.118698 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee2e32-b381-4251-9038-ac14fabbf3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.119044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee2e32-b381-4251-9038-ac14fabbf3b2 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.119742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee2e32-b381-4251-9038-ac14fabbf3b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:55.121866 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee2e32-b381-4251-9038-ac14fabbf3b2 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:55.130064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a32db-ecdf-4cf2-8db7-575327270575 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:55.146611 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ddf25-abc9-472e-bb17-83c37d64bdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:55.151077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ddf25-abc9-472e-bb17-83c37d64bdeb admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.151539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ddf25-abc9-472e-bb17-83c37d64bdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.151823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ddf25-abc9-472e-bb17-83c37d64bdeb admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.152562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ddf25-abc9-472e-bb17-83c37d64bdeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:55.153849 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1058/2103] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:54 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:55:55.156202 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ddf25-abc9-472e-bb17-83c37d64bdeb admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:55.159178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee2e32-b381-4251-9038-ac14fabbf3b2 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:55.170673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1046/2104] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:55.171950 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ddf25-abc9-472e-bb17-83c37d64bdeb admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:55.176181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee2e32-b381-4251-9038-ac14fabbf3b2 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:55.182663 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c94b8-d940-4be6-a765-3e9d5f2a2dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:55.183807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c94b8-d940-4be6-a765-3e9d5f2a2dbb admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.184203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c94b8-d940-4be6-a765-3e9d5f2a2dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.184553 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c94b8-d940-4be6-a765-3e9d5f2a2dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.185486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c94b8-d940-4be6-a765-3e9d5f2a2dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:55.187850 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c94b8-d940-4be6-a765-3e9d5f2a2dbb admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:55.188843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ddf25-abc9-472e-bb17-83c37d64bdeb admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:55.195297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57856ddc-1653-43b7-8e50-8a7a9074903f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:55.196170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57856ddc-1653-43b7-8e50-8a7a9074903f admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.196444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57856ddc-1653-43b7-8e50-8a7a9074903f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.196739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57856ddc-1653-43b7-8e50-8a7a9074903f admin admin] PATH_INFO: `/v3/projects/87087c604e604836ab98365c0b2868c8/users/5c4cf5ea0d3b4a0499588855ac24b2bf/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.218985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57856ddc-1653-43b7-8e50-8a7a9074903f admin admin] RBAC: Authorizing `identity:create_grant(project_id=87087c604e604836ab98365c0b2868c8, user_id=5c4cf5ea0d3b4a0499588855ac24b2bf, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:55.222771 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1059/2105] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:55:55.222939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57856ddc-1653-43b7-8e50-8a7a9074903f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:55.223890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c94b8-d940-4be6-a765-3e9d5f2a2dbb admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:55.240355 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:55.244590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1047/2106] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:55 2022] PUT /identity/v3/projects/87087c604e604836ab98365c0b2868c8/users/5c4cf5ea0d3b4a0499588855ac24b2bf/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:55.248730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c94b8-d940-4be6-a765-3e9d5f2a2dbb admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:55.249699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57856ddc-1653-43b7-8e50-8a7a9074903f admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:55.331815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9ba1f-7802-45b9-93f7-8b229fbe271e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:55.333054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9ba1f-7802-45b9-93f7-8b229fbe271e admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.333480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9ba1f-7802-45b9-93f7-8b229fbe271e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.333787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9ba1f-7802-45b9-93f7-8b229fbe271e admin admin] PATH_INFO: `/v3/projects/71aaf79af08a4764b3014761d0d79978/users/685d43151aea418b904a4c182da75677/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.359586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9ba1f-7802-45b9-93f7-8b229fbe271e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71aaf79af08a4764b3014761d0d79978, user_id=685d43151aea418b904a4c182da75677, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:55.364060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57856ddc-1653-43b7-8e50-8a7a9074903f admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:55.370030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4a03e-0201-42fd-a536-69dea122ca10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:55.370786 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4a03e-0201-42fd-a536-69dea122ca10 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.372186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4a03e-0201-42fd-a536-69dea122ca10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.372186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4a03e-0201-42fd-a536-69dea122ca10 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.372693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4a03e-0201-42fd-a536-69dea122ca10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:55.375359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4a03e-0201-42fd-a536-69dea122ca10 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:55.376233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9ba1f-7802-45b9-93f7-8b229fbe271e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:55.397825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1048/2107] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:55.407398 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4a03e-0201-42fd-a536-69dea122ca10 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:55.417697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1060/2108] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:55 2022] PUT /identity/v3/projects/71aaf79af08a4764b3014761d0d79978/users/685d43151aea418b904a4c182da75677/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:55.418754 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9ba1f-7802-45b9-93f7-8b229fbe271e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:55.422423 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4a03e-0201-42fd-a536-69dea122ca10 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:55.487843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:55.555946 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062c9dc-2533-4f3f-8638-2dd967542666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:55.556842 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062c9dc-2533-4f3f-8638-2dd967542666 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.558108 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062c9dc-2533-4f3f-8638-2dd967542666 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.558496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062c9dc-2533-4f3f-8638-2dd967542666 admin admin] PATH_INFO: `/v3/projects/87087c604e604836ab98365c0b2868c8/users/5c4cf5ea0d3b4a0499588855ac24b2bf/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.559620 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9ba1f-7802-45b9-93f7-8b229fbe271e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:55.567969 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062c9dc-2533-4f3f-8638-2dd967542666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87087c604e604836ab98365c0b2868c8, user_id=5c4cf5ea0d3b4a0499588855ac24b2bf, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:55.570477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59091df2-0b07-4e73-9263-fa4acf863800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:55.571028 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062c9dc-2533-4f3f-8638-2dd967542666 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:55.571422 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59091df2-0b07-4e73-9263-fa4acf863800 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.571833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59091df2-0b07-4e73-9263-fa4acf863800 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.572276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59091df2-0b07-4e73-9263-fa4acf863800 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.573335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59091df2-0b07-4e73-9263-fa4acf863800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:55.575909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59091df2-0b07-4e73-9263-fa4acf863800 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:55.593249 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1049/2109] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:55 2022] PUT /identity/v3/projects/87087c604e604836ab98365c0b2868c8/users/5c4cf5ea0d3b4a0499588855ac24b2bf/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:55.593477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1061/2110] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:55.599491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59091df2-0b07-4e73-9263-fa4acf863800 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:55.602662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06f7af-9c47-4174-9cad-572323aba30e None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.602662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06f7af-9c47-4174-9cad-572323aba30e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.602662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06f7af-9c47-4174-9cad-572323aba30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.610828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59091df2-0b07-4e73-9263-fa4acf863800 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:55.683514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-ab06f7af-9c47-4174-9cad-572323aba30e None None] MFA Rules not processed for user `5c4cf5ea0d3b4a0499588855ac24b2bf`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:55.685193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ab06f7af-9c47-4174-9cad-572323aba30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:55.687435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbc285-4271-4eae-91fe-f529b8810119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:55.688425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbc285-4271-4eae-91fe-f529b8810119 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.688786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbc285-4271-4eae-91fe-f529b8810119 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.689404 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbc285-4271-4eae-91fe-f529b8810119 admin admin] PATH_INFO: `/v3/projects/71aaf79af08a4764b3014761d0d79978/users/685d43151aea418b904a4c182da75677/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.691469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbc285-4271-4eae-91fe-f529b8810119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71aaf79af08a4764b3014761d0d79978, user_id=685d43151aea418b904a4c182da75677, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:55.694743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbc285-4271-4eae-91fe-f529b8810119 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:55.708439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1062/2111] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:55 2022] PUT /identity/v3/projects/71aaf79af08a4764b3014761d0d79978/users/685d43151aea418b904a4c182da75677/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:55.727338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77feb0a9-154b-403f-8d7c-91452995c36a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.727626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77feb0a9-154b-403f-8d7c-91452995c36a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.727913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77feb0a9-154b-403f-8d7c-91452995c36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.760415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-77feb0a9-154b-403f-8d7c-91452995c36a None None] MFA Rules not processed for user `685d43151aea418b904a4c182da75677`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:55.761784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-77feb0a9-154b-403f-8d7c-91452995c36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:55.805790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1050/2112] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:55.818327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06f7af-9c47-4174-9cad-572323aba30e None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:55.820208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ab06f7af-9c47-4174-9cad-572323aba30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:55.843429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:55.864314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1063/2113] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:55.872411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77feb0a9-154b-403f-8d7c-91452995c36a None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:55.874569 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-77feb0a9-154b-403f-8d7c-91452995c36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:55.903178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06f7af-9c47-4174-9cad-572323aba30e None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:55.937263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:55.950576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77feb0a9-154b-403f-8d7c-91452995c36a None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:55.955561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc088334-7065-4de8-8087-632ee7c34f16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:55.956289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc088334-7065-4de8-8087-632ee7c34f16 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.956820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc088334-7065-4de8-8087-632ee7c34f16 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.957173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc088334-7065-4de8-8087-632ee7c34f16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.962651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf65b67-92b5-4200-881a-bad8c20fd9d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:55.963266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf65b67-92b5-4200-881a-bad8c20fd9d4 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:55.963549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf65b67-92b5-4200-881a-bad8c20fd9d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:55.963821 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf65b67-92b5-4200-881a-bad8c20fd9d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:55.968239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc088334-7065-4de8-8087-632ee7c34f16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:55.969753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc088334-7065-4de8-8087-632ee7c34f16 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:55.990314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1064/2114] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:56.035755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf65b67-92b5-4200-881a-bad8c20fd9d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:56.037390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf65b67-92b5-4200-881a-bad8c20fd9d4 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:56.049291 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1051/2115] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:55:56.087914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc088334-7065-4de8-8087-632ee7c34f16 service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:56.097167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc088334-7065-4de8-8087-632ee7c34f16 service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:56.102575 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6ace2-7e04-4b5d-a63e-e9bc44a36bb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:56.103234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6ace2-7e04-4b5d-a63e-e9bc44a36bb4 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:56.103516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6ace2-7e04-4b5d-a63e-e9bc44a36bb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:56.103789 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6ace2-7e04-4b5d-a63e-e9bc44a36bb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:56.142253 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf65b67-92b5-4200-881a-bad8c20fd9d4 service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:56.150890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf65b67-92b5-4200-881a-bad8c20fd9d4 service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:56.155294 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96b791-1b37-4040-a8fb-c508f408a9d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:56.155953 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96b791-1b37-4040-a8fb-c508f408a9d3 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:56.156236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96b791-1b37-4040-a8fb-c508f408a9d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:56.156515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96b791-1b37-4040-a8fb-c508f408a9d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:56.167038 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6ace2-7e04-4b5d-a63e-e9bc44a36bb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:56.168499 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6ace2-7e04-4b5d-a63e-e9bc44a36bb4 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:56.178911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1065/2116] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:56.225617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96b791-1b37-4040-a8fb-c508f408a9d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:56.227083 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96b791-1b37-4040-a8fb-c508f408a9d3 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:56.237092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1052/2117] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:56.565590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6ace2-7e04-4b5d-a63e-e9bc44a36bb4 service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:56.630700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6ace2-7e04-4b5d-a63e-e9bc44a36bb4 service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:56.635384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:56.636241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:56.636525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:56.636802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] PATH_INFO: `/v3/users/22b56ce385f240bfa9ce353747af4ae6` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:56.637984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22b56ce385f240bfa9ce353747af4ae6)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:56.640692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:56.686352 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22b56ce385f240bfa9ce353747af4ae6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:56.693913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22b56ce385f240bfa9ce353747af4ae6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:56.702807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22b56ce385f240bfa9ce353747af4ae6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:56.703495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22b56ce385f240bfa9ce353747af4ae6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:56.704111 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22b56ce385f240bfa9ce353747af4ae6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:56.721288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1066/2118] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:56 2022] DELETE /identity/v3/users/22b56ce385f240bfa9ce353747af4ae6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:56.725819 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d6e38-36ad-486a-a418-07ce622eebed None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:56.726139 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d6e38-36ad-486a-a418-07ce622eebed None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:56.726420 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d6e38-36ad-486a-a418-07ce622eebed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:56.744323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-245d6e38-36ad-486a-a418-07ce622eebed None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:56.745520 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-245d6e38-36ad-486a-a418-07ce622eebed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:56.758318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:56.814824 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1053/2119] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:56.826078 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:56.828209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:56.834841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:56.895674 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbb1e8-c539-461f-8b59-1cba07b76af9 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:56.899944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6222e-91d3-41ce-ab1f-db908aaa605d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:56.900599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6222e-91d3-41ce-ab1f-db908aaa605d service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:56.900887 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6222e-91d3-41ce-ab1f-db908aaa605d service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:56.901280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6222e-91d3-41ce-ab1f-db908aaa605d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:56.915252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6222e-91d3-41ce-ab1f-db908aaa605d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:56.916980 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6222e-91d3-41ce-ab1f-db908aaa605d service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:56.927862 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1067/2120] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:57.336592 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d6e38-36ad-486a-a418-07ce622eebed None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:57.338390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-245d6e38-36ad-486a-a418-07ce622eebed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:57.358917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d6e38-36ad-486a-a418-07ce622eebed None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:57.371386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:57.372336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:57.372677 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:57.373511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] PATH_INFO: `/v3/projects/763e30960daa4e638606477f656ae74c` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:57.375404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=763e30960daa4e638606477f656ae74c)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:57.377384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:57.438855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '763e30960daa4e638606477f656ae74c', 'request_id': 'req-72fc64f9-37e8-445d-9b50-08acf0f27493'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:57.444929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '763e30960daa4e638606477f656ae74c', 'request_id': 'req-72fc64f9-37e8-445d-9b50-08acf0f27493'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:57.452196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '763e30960daa4e638606477f656ae74c', 'request_id': 'req-72fc64f9-37e8-445d-9b50-08acf0f27493'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:57.462723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] The token cache is being invalidate because project 763e30960daa4e638606477f656ae74c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:57.463043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:57.463672 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:57.465221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1054/2121] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:57 2022] DELETE /identity/v3/projects/763e30960daa4e638606477f656ae74c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:57.469586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6222e-91d3-41ce-ab1f-db908aaa605d service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:57.480048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6222e-91d3-41ce-ab1f-db908aaa605d service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:57.538999 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ef33f-41c6-4784-85ac-f2df0f4cf9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:57.539970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ef33f-41c6-4784-85ac-f2df0f4cf9af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:57.540210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ef33f-41c6-4784-85ac-f2df0f4cf9af admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:57.540511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ef33f-41c6-4784-85ac-f2df0f4cf9af admin admin] PATH_INFO: `/v3/users/5c4cf5ea0d3b4a0499588855ac24b2bf` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:57.541722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ef33f-41c6-4784-85ac-f2df0f4cf9af admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c4cf5ea0d3b4a0499588855ac24b2bf)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:57.543689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ef33f-41c6-4784-85ac-f2df0f4cf9af admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:57.612741 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3b8ef33f-41c6-4784-85ac-f2df0f4cf9af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c4cf5ea0d3b4a0499588855ac24b2bf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:57.622485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3b8ef33f-41c6-4784-85ac-f2df0f4cf9af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c4cf5ea0d3b4a0499588855ac24b2bf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:57.643677 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3b8ef33f-41c6-4784-85ac-f2df0f4cf9af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c4cf5ea0d3b4a0499588855ac24b2bf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:57.643677 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3b8ef33f-41c6-4784-85ac-f2df0f4cf9af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c4cf5ea0d3b4a0499588855ac24b2bf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:57.643677 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3b8ef33f-41c6-4784-85ac-f2df0f4cf9af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c4cf5ea0d3b4a0499588855ac24b2bf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:57.679306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1068/2122] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:57 2022] DELETE /identity/v3/users/5c4cf5ea0d3b4a0499588855ac24b2bf => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:57.981199 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:57.983935 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:58.001669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc64f9-37e8-445d-9b50-08acf0f27493 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:58.073120 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75c7af-0d5e-4391-805d-78db38d9012b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:58.074017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75c7af-0d5e-4391-805d-78db38d9012b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:58.074289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75c7af-0d5e-4391-805d-78db38d9012b admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:58.074559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75c7af-0d5e-4391-805d-78db38d9012b admin admin] PATH_INFO: `/v3/projects/87087c604e604836ab98365c0b2868c8` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:58.076674 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f75c7af-0d5e-4391-805d-78db38d9012b admin admin] RBAC: Authorizing `identity:delete_project(project_id=87087c604e604836ab98365c0b2868c8)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:58.078624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f75c7af-0d5e-4391-805d-78db38d9012b admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:58.126109 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:58.131055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2f75c7af-0d5e-4391-805d-78db38d9012b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87087c604e604836ab98365c0b2868c8', 'request_id': 'req-2f75c7af-0d5e-4391-805d-78db38d9012b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:58.138495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2f75c7af-0d5e-4391-805d-78db38d9012b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87087c604e604836ab98365c0b2868c8', 'request_id': 'req-2f75c7af-0d5e-4391-805d-78db38d9012b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:58.147717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2f75c7af-0d5e-4391-805d-78db38d9012b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87087c604e604836ab98365c0b2868c8', 'request_id': 'req-2f75c7af-0d5e-4391-805d-78db38d9012b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:58.160467 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2f75c7af-0d5e-4391-805d-78db38d9012b admin admin] The token cache is being invalidate because project 87087c604e604836ab98365c0b2868c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:55:58.160748 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2f75c7af-0d5e-4391-805d-78db38d9012b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:58.161463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2f75c7af-0d5e-4391-805d-78db38d9012b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:58.163048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1055/2123] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:57 2022] DELETE /identity/v3/projects/87087c604e604836ab98365c0b2868c8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:58.175022 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb24af-169d-4221-b60c-b2898d05cc09 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:58.177254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb24af-169d-4221-b60c-b2898d05cc09 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:58.177548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb24af-169d-4221-b60c-b2898d05cc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:58.198241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-bfeb24af-169d-4221-b60c-b2898d05cc09 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:58.199808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-bfeb24af-169d-4221-b60c-b2898d05cc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:58.221518 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:58.269057 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1069/2124] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:58.277762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb907a4a-e351-482c-ba2a-77cf63c58fdd None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:58.277915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb907a4a-e351-482c-ba2a-77cf63c58fdd None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:58.278168 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb907a4a-e351-482c-ba2a-77cf63c58fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:58.300051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-eb907a4a-e351-482c-ba2a-77cf63c58fdd None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:58.301445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-eb907a4a-e351-482c-ba2a-77cf63c58fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:58.313935 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1056/2125] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:58.318284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb24af-169d-4221-b60c-b2898d05cc09 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:58.328572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb24af-169d-4221-b60c-b2898d05cc09 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:58.333060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edd9fa-2fc6-42d3-bf4a-9b77a62d1856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:58.333710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edd9fa-2fc6-42d3-bf4a-9b77a62d1856 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:58.333998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edd9fa-2fc6-42d3-bf4a-9b77a62d1856 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:58.334275 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edd9fa-2fc6-42d3-bf4a-9b77a62d1856 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:58.335037 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edd9fa-2fc6-42d3-bf4a-9b77a62d1856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:55:58.335347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edd9fa-2fc6-42d3-bf4a-9b77a62d1856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:58.337000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edd9fa-2fc6-42d3-bf4a-9b77a62d1856 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:58.350906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1070/2126] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:55:58.355288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb907a4a-e351-482c-ba2a-77cf63c58fdd None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:58.364726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb907a4a-e351-482c-ba2a-77cf63c58fdd None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:58.369452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41093cb4-0390-476c-bc2c-943e17dbab71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:58.370245 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41093cb4-0390-476c-bc2c-943e17dbab71 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:58.370713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41093cb4-0390-476c-bc2c-943e17dbab71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:58.371020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41093cb4-0390-476c-bc2c-943e17dbab71 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:58.372014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41093cb4-0390-476c-bc2c-943e17dbab71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:58.375047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41093cb4-0390-476c-bc2c-943e17dbab71 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:58.407169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1057/2127] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:55:58 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:55:58.412013 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edd9fa-2fc6-42d3-bf4a-9b77a62d1856 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:58.423437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edd9fa-2fc6-42d3-bf4a-9b77a62d1856 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:58.492365 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd811f4-8d2a-4283-9fb1-2a83c9515382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:58.493149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd811f4-8d2a-4283-9fb1-2a83c9515382 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:58.493525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd811f4-8d2a-4283-9fb1-2a83c9515382 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:58.493859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd811f4-8d2a-4283-9fb1-2a83c9515382 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:58.494625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd811f4-8d2a-4283-9fb1-2a83c9515382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:58.496690 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd811f4-8d2a-4283-9fb1-2a83c9515382 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:58.523944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1071/2128] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:55:58 2022] POST /identity/v3/users => generated 285 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:55:58.527930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41093cb4-0390-476c-bc2c-943e17dbab71 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:58.539373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41093cb4-0390-476c-bc2c-943e17dbab71 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:58.544356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6f793-8b01-4f53-9618-cdb013770aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:58.545622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6f793-8b01-4f53-9618-cdb013770aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:58.546138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6f793-8b01-4f53-9618-cdb013770aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:58.546503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6f793-8b01-4f53-9618-cdb013770aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:58.547348 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6f793-8b01-4f53-9618-cdb013770aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:58.549063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6f793-8b01-4f53-9618-cdb013770aa7 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:58.560593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1058/2129] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:58.565029 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd811f4-8d2a-4283-9fb1-2a83c9515382 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:58.575243 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd811f4-8d2a-4283-9fb1-2a83c9515382 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:58.581093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836682-3cf6-4bba-bbc9-a677bd0cc84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:58.581685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836682-3cf6-4bba-bbc9-a677bd0cc84b admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:58.581967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836682-3cf6-4bba-bbc9-a677bd0cc84b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:58.582241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836682-3cf6-4bba-bbc9-a677bd0cc84b admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:58.583000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836682-3cf6-4bba-bbc9-a677bd0cc84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:58.584585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836682-3cf6-4bba-bbc9-a677bd0cc84b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:58.595713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1072/2130] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:55:58.599702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6f793-8b01-4f53-9618-cdb013770aa7 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:58.612813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6f793-8b01-4f53-9618-cdb013770aa7 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:58.617623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17d05c-cde8-4c3f-b910-98842a60b01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:58.618507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17d05c-cde8-4c3f-b910-98842a60b01d admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:58.618785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17d05c-cde8-4c3f-b910-98842a60b01d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:58.619088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17d05c-cde8-4c3f-b910-98842a60b01d admin admin] PATH_INFO: `/v3/projects/957af36a92db42a19f014ab2948e5516/users/b7f3493e90ae4776986b862237846099/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:58.635412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17d05c-cde8-4c3f-b910-98842a60b01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=957af36a92db42a19f014ab2948e5516, user_id=b7f3493e90ae4776986b862237846099, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:58.638094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17d05c-cde8-4c3f-b910-98842a60b01d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:58.655018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1059/2131] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:55:58 2022] PUT /identity/v3/projects/957af36a92db42a19f014ab2948e5516/users/b7f3493e90ae4776986b862237846099/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:55:58.665529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd1f55-f219-4e54-a679-48210e5db640 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:58.666035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd1f55-f219-4e54-a679-48210e5db640 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:58.666322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd1f55-f219-4e54-a679-48210e5db640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:58.697300 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-cecd1f55-f219-4e54-a679-48210e5db640 None None] MFA Rules not processed for user `b7f3493e90ae4776986b862237846099`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:58.698545 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-cecd1f55-f219-4e54-a679-48210e5db640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:58.771812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1073/2132] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:58.784831 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17d05c-cde8-4c3f-b910-98842a60b01d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:58.786552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ef17d05c-cde8-4c3f-b910-98842a60b01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:58.851944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd1f55-f219-4e54-a679-48210e5db640 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:58.853369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-cecd1f55-f219-4e54-a679-48210e5db640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:58.856590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17d05c-cde8-4c3f-b910-98842a60b01d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:58.861302 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ee94b-c470-427d-b517-80161a6d8f5b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '666ade4ce1fe447487998d086779389b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:58.861959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ee94b-c470-427d-b517-80161a6d8f5b service glance] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:58.862255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ee94b-c470-427d-b517-80161a6d8f5b service glance] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:58.862540 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ee94b-c470-427d-b517-80161a6d8f5b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:58.873905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ee94b-c470-427d-b517-80161a6d8f5b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:58.875823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ee94b-c470-427d-b517-80161a6d8f5b service glance] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:58.882235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:55:58.887963 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1060/2133] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:55:58.932299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd1f55-f219-4e54-a679-48210e5db640 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:58.937406 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:58.938318 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:58.938648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:58.938923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] PATH_INFO: `/v3/users/028c23493a524f14a5996719ad1b9448` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:58.943794 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=028c23493a524f14a5996719ad1b9448)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:58.945824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:59.004515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '028c23493a524f14a5996719ad1b9448'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:59.012506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '028c23493a524f14a5996719ad1b9448'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:59.023041 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '028c23493a524f14a5996719ad1b9448'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:59.023974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '028c23493a524f14a5996719ad1b9448'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:59.024672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '028c23493a524f14a5996719ad1b9448'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:55:59.047672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1074/2134] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:55:58 2022] DELETE /identity/v3/users/028c23493a524f14a5996719ad1b9448 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:55:59.052552 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4484fb-f707-442a-ad1c-9f5f566ca777 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:59.052852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4484fb-f707-442a-ad1c-9f5f566ca777 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:59.053145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4484fb-f707-442a-ad1c-9f5f566ca777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:59.076857 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-ba4484fb-f707-442a-ad1c-9f5f566ca777 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:55:59.078188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ba4484fb-f707-442a-ad1c-9f5f566ca777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:59.161667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1061/2135] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:55:59.171965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:59.173436 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:59.257133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d59fb-d52a-4d9a-89e8-c4a98bcb0671 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:59.261914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9da0f-3f76-439b-a048-a9256268ad21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:59.262651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9da0f-3f76-439b-a048-a9256268ad21 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:59.264316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9da0f-3f76-439b-a048-a9256268ad21 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:59.264316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9da0f-3f76-439b-a048-a9256268ad21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:59.276176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9da0f-3f76-439b-a048-a9256268ad21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:59.278218 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9da0f-3f76-439b-a048-a9256268ad21 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:59.293738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1075/2136] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:55:59.897058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4484fb-f707-442a-ad1c-9f5f566ca777 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:55:59.898585 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ba4484fb-f707-442a-ad1c-9f5f566ca777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:55:59.913842 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4484fb-f707-442a-ad1c-9f5f566ca777 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:55:59.920675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:55:59.922013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:55:59.922453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:55:59.923016 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] PATH_INFO: `/v3/projects/d698f3c70e30495c8c3ab15afb488784` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:55:59.924960 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d698f3c70e30495c8c3ab15afb488784)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:55:59.927705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:55:59.936996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:00.033801 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd698f3c70e30495c8c3ab15afb488784', 'request_id': 'req-e366b6c6-f749-4cdf-8f54-0ccc496cad58'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.041988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd698f3c70e30495c8c3ab15afb488784', 'request_id': 'req-e366b6c6-f749-4cdf-8f54-0ccc496cad58'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.050932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd698f3c70e30495c8c3ab15afb488784', 'request_id': 'req-e366b6c6-f749-4cdf-8f54-0ccc496cad58'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.069280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] The token cache is being invalidate because project d698f3c70e30495c8c3ab15afb488784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:00.069571 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.071270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.073278 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1062/2137] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:55:59 2022] DELETE /identity/v3/projects/d698f3c70e30495c8c3ab15afb488784 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:00.077533 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9da0f-3f76-439b-a048-a9256268ad21 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:00.090319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9da0f-3f76-439b-a048-a9256268ad21 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:00.127117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:00.128582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:00.148260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:00.168132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:00.169020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:00.169312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:00.169589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] PATH_INFO: `/v3/users/685d43151aea418b904a4c182da75677` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:00.171858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=685d43151aea418b904a4c182da75677)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:00.173835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:00.205323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366b6c6-f749-4cdf-8f54-0ccc496cad58 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:00.210986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:00.211950 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:00.212237 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:00.212513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] PATH_INFO: `/v3/users/8fca4ff548a948d09976d7cdfa6ad902` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:00.213768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fca4ff548a948d09976d7cdfa6ad902)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:00.215805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:00.228583 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '685d43151aea418b904a4c182da75677'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.239004 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '685d43151aea418b904a4c182da75677'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.249577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '685d43151aea418b904a4c182da75677'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.256110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '685d43151aea418b904a4c182da75677'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.256778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '685d43151aea418b904a4c182da75677'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.277044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fca4ff548a948d09976d7cdfa6ad902'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.280900 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1076/2138] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:00 2022] DELETE /identity/v3/users/685d43151aea418b904a4c182da75677 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:00.284777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fca4ff548a948d09976d7cdfa6ad902'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.285480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fca4ff548a948d09976d7cdfa6ad902'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.293517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fca4ff548a948d09976d7cdfa6ad902'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.306533 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fca4ff548a948d09976d7cdfa6ad902'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.316888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1063/2139] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:00 2022] DELETE /identity/v3/users/8fca4ff548a948d09976d7cdfa6ad902 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:00.632022 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:00.633558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:00.719085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:00.721522 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:00.747607 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9110cee-757f-41d1-a929-ac6b4e349150 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:00.757181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:00.834093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f79410-5a21-4ef8-b06d-dc986d559314 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:00.845199 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:00.846085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:00.846373 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:00.846653 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5 admin admin] PATH_INFO: `/v3/projects/71aaf79af08a4764b3014761d0d79978` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:00.847906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71aaf79af08a4764b3014761d0d79978)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:00.849987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:00.851838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:00.852715 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:00.852998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:00.853376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] PATH_INFO: `/v3/projects/8cd6cd96f64a43d79ed6c5178275a1a8` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:00.854481 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd6cd96f64a43d79ed6c5178275a1a8)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:00.856414 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:00.869246 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:00.908474 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71aaf79af08a4764b3014761d0d79978', 'request_id': 'req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.916520 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71aaf79af08a4764b3014761d0d79978', 'request_id': 'req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.923322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd6cd96f64a43d79ed6c5178275a1a8', 'request_id': 'req-a67dcc1b-7655-4af2-9512-cb817d01d599'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.927806 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71aaf79af08a4764b3014761d0d79978', 'request_id': 'req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.931062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd6cd96f64a43d79ed6c5178275a1a8', 'request_id': 'req-a67dcc1b-7655-4af2-9512-cb817d01d599'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.939855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd6cd96f64a43d79ed6c5178275a1a8', 'request_id': 'req-a67dcc1b-7655-4af2-9512-cb817d01d599'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.946749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5 admin admin] The token cache is being invalidate because project 71aaf79af08a4764b3014761d0d79978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:00.947082 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.947745 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9bd7acdc-c67e-45ad-a4c1-b70d3854f3a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.949388 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1064/2140] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:00 2022] DELETE /identity/v3/projects/71aaf79af08a4764b3014761d0d79978 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:56:00.953278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] The token cache is being invalidate because project 8cd6cd96f64a43d79ed6c5178275a1a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:00.953596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.954227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:00.954669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d06d8-fd4e-42ad-9937-f22c28c87b3f None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:00.955151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d06d8-fd4e-42ad-9937-f22c28c87b3f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:00.955538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d06d8-fd4e-42ad-9937-f22c28c87b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:00.955809 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1077/2141] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:00 2022] DELETE /identity/v3/projects/8cd6cd96f64a43d79ed6c5178275a1a8 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:56:00.959533 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:00.960832 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:00.976476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-244d06d8-fd4e-42ad-9937-f22c28c87b3f None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:00.977690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-244d06d8-fd4e-42ad-9937-f22c28c87b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:01.029745 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dcc1b-7655-4af2-9512-cb817d01d599 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.034580 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb5fdc-6956-478d-91bb-bcde95fa4002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.035484 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb5fdc-6956-478d-91bb-bcde95fa4002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.035764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb5fdc-6956-478d-91bb-bcde95fa4002 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.036037 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb5fdc-6956-478d-91bb-bcde95fa4002 admin admin] PATH_INFO: `/v3/users/e2f55e1762ac4af8b413db20d0c99f99` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.037257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb5fdc-6956-478d-91bb-bcde95fa4002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2f55e1762ac4af8b413db20d0c99f99)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.039327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb5fdc-6956-478d-91bb-bcde95fa4002 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.041500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1065/2142] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:56:01.050062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c07cb-8b52-4620-9b38-574975f933db None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.050350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c07cb-8b52-4620-9b38-574975f933db None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.050629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c07cb-8b52-4620-9b38-574975f933db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.069743 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-2d7c07cb-8b52-4620-9b38-574975f933db None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:01.071252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2d7c07cb-8b52-4620-9b38-574975f933db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:01.084954 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1066/2143] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:56:01.089113 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c07cb-8b52-4620-9b38-574975f933db None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.095376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e2bb5fdc-6956-478d-91bb-bcde95fa4002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2f55e1762ac4af8b413db20d0c99f99'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:01.099116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c07cb-8b52-4620-9b38-574975f933db None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.103866 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e2bb5fdc-6956-478d-91bb-bcde95fa4002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2f55e1762ac4af8b413db20d0c99f99'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:01.104622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28147041-881b-4d89-a0cf-21bc68a0e264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.105271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28147041-881b-4d89-a0cf-21bc68a0e264 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.105554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28147041-881b-4d89-a0cf-21bc68a0e264 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.105828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28147041-881b-4d89-a0cf-21bc68a0e264 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.106543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28147041-881b-4d89-a0cf-21bc68a0e264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:56:01.106847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28147041-881b-4d89-a0cf-21bc68a0e264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.109214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28147041-881b-4d89-a0cf-21bc68a0e264 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.112462 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e2bb5fdc-6956-478d-91bb-bcde95fa4002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2f55e1762ac4af8b413db20d0c99f99'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:01.113171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e2bb5fdc-6956-478d-91bb-bcde95fa4002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2f55e1762ac4af8b413db20d0c99f99'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:01.114096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e2bb5fdc-6956-478d-91bb-bcde95fa4002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2f55e1762ac4af8b413db20d0c99f99'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:01.125964 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1067/2144] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:56:01.127404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28147041-881b-4d89-a0cf-21bc68a0e264 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.129714 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-28147041-881b-4d89-a0cf-21bc68a0e264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:01.134853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1078/2145] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:00 2022] DELETE /identity/v3/users/e2f55e1762ac4af8b413db20d0c99f99 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:56:01.144227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28147041-881b-4d89-a0cf-21bc68a0e264 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.165704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:01.210024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92750382-a759-4354-8bc3-328dda0126ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.210840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92750382-a759-4354-8bc3-328dda0126ed admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.211254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92750382-a759-4354-8bc3-328dda0126ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.211628 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92750382-a759-4354-8bc3-328dda0126ed admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.212431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92750382-a759-4354-8bc3-328dda0126ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.214397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92750382-a759-4354-8bc3-328dda0126ed admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.240116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1068/2146] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:56:01 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:56:01.244586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb5fdc-6956-478d-91bb-bcde95fa4002 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.256608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb5fdc-6956-478d-91bb-bcde95fa4002 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.274782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:01.327240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5cb743-2fa2-4917-9993-e9914d0c639f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.327952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cb743-2fa2-4917-9993-e9914d0c639f admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.328227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cb743-2fa2-4917-9993-e9914d0c639f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.328509 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cb743-2fa2-4917-9993-e9914d0c639f admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.329338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5cb743-2fa2-4917-9993-e9914d0c639f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.331204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5cb743-2fa2-4917-9993-e9914d0c639f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.331710 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92750382-a759-4354-8bc3-328dda0126ed admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.333754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-92750382-a759-4354-8bc3-328dda0126ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:01.348637 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92750382-a759-4354-8bc3-328dda0126ed admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.354276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05a281-2991-4e33-92cc-f34dc8398be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.355463 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1079/2147] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:56:01 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:56:01.355585 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05a281-2991-4e33-92cc-f34dc8398be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.355833 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05a281-2991-4e33-92cc-f34dc8398be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.356208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05a281-2991-4e33-92cc-f34dc8398be8 admin admin] PATH_INFO: `/v3/projects/d219e5c6f71a47c1b17ff0f68bee44e4` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.358183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05a281-2991-4e33-92cc-f34dc8398be8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d219e5c6f71a47c1b17ff0f68bee44e4)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.360416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5cb743-2fa2-4917-9993-e9914d0c639f admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.360550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05a281-2991-4e33-92cc-f34dc8398be8 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.371797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5cb743-2fa2-4917-9993-e9914d0c639f admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.377875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f0387-9a6b-481d-a4b2-fc9870d7fbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.378975 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f0387-9a6b-481d-a4b2-fc9870d7fbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.379270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f0387-9a6b-481d-a4b2-fc9870d7fbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.379554 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f0387-9a6b-481d-a4b2-fc9870d7fbd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.380293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f0387-9a6b-481d-a4b2-fc9870d7fbd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.381899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f0387-9a6b-481d-a4b2-fc9870d7fbd0 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.399312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1080/2148] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:56:01.404072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f0387-9a6b-481d-a4b2-fc9870d7fbd0 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.417770 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f0387-9a6b-481d-a4b2-fc9870d7fbd0 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.433786 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4e05a281-2991-4e33-92cc-f34dc8398be8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd219e5c6f71a47c1b17ff0f68bee44e4', 'request_id': 'req-4e05a281-2991-4e33-92cc-f34dc8398be8'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:01.442516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4e05a281-2991-4e33-92cc-f34dc8398be8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd219e5c6f71a47c1b17ff0f68bee44e4', 'request_id': 'req-4e05a281-2991-4e33-92cc-f34dc8398be8'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:01.450670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4e05a281-2991-4e33-92cc-f34dc8398be8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd219e5c6f71a47c1b17ff0f68bee44e4', 'request_id': 'req-4e05a281-2991-4e33-92cc-f34dc8398be8'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:01.468730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4e05a281-2991-4e33-92cc-f34dc8398be8 admin admin] The token cache is being invalidate because project d219e5c6f71a47c1b17ff0f68bee44e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:01.469361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4e05a281-2991-4e33-92cc-f34dc8398be8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:01.470574 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4e05a281-2991-4e33-92cc-f34dc8398be8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:01.473299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1069/2149] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:01 2022] DELETE /identity/v3/projects/d219e5c6f71a47c1b17ff0f68bee44e4 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:56:01.478049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2630df8-ac09-4db3-98bd-ea95feec1f79 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.478346 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2630df8-ac09-4db3-98bd-ea95feec1f79 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.478691 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2630df8-ac09-4db3-98bd-ea95feec1f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.486541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdb2b6-95a4-4279-bb4a-95bd876f7722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.487261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdb2b6-95a4-4279-bb4a-95bd876f7722 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.487542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdb2b6-95a4-4279-bb4a-95bd876f7722 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.487816 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdb2b6-95a4-4279-bb4a-95bd876f7722 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.488543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdb2b6-95a4-4279-bb4a-95bd876f7722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.490133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdb2b6-95a4-4279-bb4a-95bd876f7722 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.499853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-e2630df8-ac09-4db3-98bd-ea95feec1f79 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:01.502097 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e2630df8-ac09-4db3-98bd-ea95feec1f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:01.511110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1070/2150] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:56:01.518623 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1081/2151] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:01.520781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac91ba8-d137-4abb-a963-d2b3acee2c6a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.521154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac91ba8-d137-4abb-a963-d2b3acee2c6a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.521517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac91ba8-d137-4abb-a963-d2b3acee2c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.522680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdb2b6-95a4-4279-bb4a-95bd876f7722 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.525517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-66bdb2b6-95a4-4279-bb4a-95bd876f7722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:01.544714 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdb2b6-95a4-4279-bb4a-95bd876f7722 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.546518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-fac91ba8-d137-4abb-a963-d2b3acee2c6a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:01.548201 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-fac91ba8-d137-4abb-a963-d2b3acee2c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:01.556664 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7376382-23f0-401d-80c2-9d06fa831ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.557568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7376382-23f0-401d-80c2-9d06fa831ece admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.557853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7376382-23f0-401d-80c2-9d06fa831ece admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.558130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7376382-23f0-401d-80c2-9d06fa831ece admin admin] PATH_INFO: `/v3/projects/74eb269b1a194c83ab87a80ebbfb8c86/users/7ba5493652a64aeb942c3c9c9f68d09b/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.568524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1071/2152] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:01.572607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac91ba8-d137-4abb-a963-d2b3acee2c6a None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.580381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7376382-23f0-401d-80c2-9d06fa831ece admin admin] RBAC: Authorizing `identity:create_grant(project_id=74eb269b1a194c83ab87a80ebbfb8c86, user_id=7ba5493652a64aeb942c3c9c9f68d09b, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.583171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7376382-23f0-401d-80c2-9d06fa831ece admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.584502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac91ba8-d137-4abb-a963-d2b3acee2c6a None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.590787 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be483fd4-1856-4177-bf4d-e0f14f82c097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.591563 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be483fd4-1856-4177-bf4d-e0f14f82c097 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.591896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be483fd4-1856-4177-bf4d-e0f14f82c097 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.592359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be483fd4-1856-4177-bf4d-e0f14f82c097 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.593545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be483fd4-1856-4177-bf4d-e0f14f82c097 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:56:01.593895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be483fd4-1856-4177-bf4d-e0f14f82c097 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.595564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be483fd4-1856-4177-bf4d-e0f14f82c097 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.606729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1082/2153] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:01 2022] PUT /identity/v3/projects/74eb269b1a194c83ab87a80ebbfb8c86/users/7ba5493652a64aeb942c3c9c9f68d09b/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:56:01.609063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1072/2154] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:56:01.613393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7376382-23f0-401d-80c2-9d06fa831ece admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.621007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a27bd0-9624-4896-856c-99c72d2aa735 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.621502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a27bd0-9624-4896-856c-99c72d2aa735 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.621787 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a27bd0-9624-4896-856c-99c72d2aa735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.652663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-d0a27bd0-9624-4896-856c-99c72d2aa735 None None] MFA Rules not processed for user `7ba5493652a64aeb942c3c9c9f68d09b`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:01.653888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d0a27bd0-9624-4896-856c-99c72d2aa735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:01.681493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7376382-23f0-401d-80c2-9d06fa831ece admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.686591 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eac028-da46-4333-ad04-80ba57457d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.687299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eac028-da46-4333-ad04-80ba57457d32 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.687589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eac028-da46-4333-ad04-80ba57457d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.687925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eac028-da46-4333-ad04-80ba57457d32 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.688700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eac028-da46-4333-ad04-80ba57457d32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.690665 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eac028-da46-4333-ad04-80ba57457d32 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.705446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:01.712108 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:01.723279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1083/2155] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:56:01 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:56:01.727718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eac028-da46-4333-ad04-80ba57457d32 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.735562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1073/2156] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:56:01.736852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a27bd0-9624-4896-856c-99c72d2aa735 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.749288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a27bd0-9624-4896-856c-99c72d2aa735 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.805330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eac028-da46-4333-ad04-80ba57457d32 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.811139 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0bb91-3170-4b06-b1da-394b0dd2948c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.811963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0bb91-3170-4b06-b1da-394b0dd2948c admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.812244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0bb91-3170-4b06-b1da-394b0dd2948c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.812515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0bb91-3170-4b06-b1da-394b0dd2948c admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.813272 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0bb91-3170-4b06-b1da-394b0dd2948c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.815615 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0bb91-3170-4b06-b1da-394b0dd2948c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.820463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e45447-c903-4bc3-b7aa-aa181e6afc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.821131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e45447-c903-4bc3-b7aa-aa181e6afc7c admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.821423 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e45447-c903-4bc3-b7aa-aa181e6afc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.821707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e45447-c903-4bc3-b7aa-aa181e6afc7c admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.822379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e45447-c903-4bc3-b7aa-aa181e6afc7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.824342 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e45447-c903-4bc3-b7aa-aa181e6afc7c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.845204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1084/2157] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:56:01 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:56:01.849769 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0bb91-3170-4b06-b1da-394b0dd2948c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.870494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1074/2158] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:56:01 2022] POST /identity/v3/projects => generated 389 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:01.872028 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e45447-c903-4bc3-b7aa-aa181e6afc7c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.874317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0bb91-3170-4b06-b1da-394b0dd2948c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.944417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56003d-7572-42f1-b1f7-46faa491d442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.945097 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56003d-7572-42f1-b1f7-46faa491d442 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.945455 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56003d-7572-42f1-b1f7-46faa491d442 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.945823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56003d-7572-42f1-b1f7-46faa491d442 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.946631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56003d-7572-42f1-b1f7-46faa491d442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.948489 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56003d-7572-42f1-b1f7-46faa491d442 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.950003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e45447-c903-4bc3-b7aa-aa181e6afc7c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.956236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35924c4-ea69-4c04-9067-1aa793882cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.956982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35924c4-ea69-4c04-9067-1aa793882cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.957365 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35924c4-ea69-4c04-9067-1aa793882cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.957687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35924c4-ea69-4c04-9067-1aa793882cf7 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.958445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35924c4-ea69-4c04-9067-1aa793882cf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.959658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1085/2159] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:01.960765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35924c4-ea69-4c04-9067-1aa793882cf7 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.963597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56003d-7572-42f1-b1f7-46faa491d442 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:01.974905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56003d-7572-42f1-b1f7-46faa491d442 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:01.980764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218baeb7-8e14-4def-b236-918afa6df006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:01.981404 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218baeb7-8e14-4def-b236-918afa6df006 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:01.981686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218baeb7-8e14-4def-b236-918afa6df006 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:01.981962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218baeb7-8e14-4def-b236-918afa6df006 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:01.983284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218baeb7-8e14-4def-b236-918afa6df006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:01.985154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218baeb7-8e14-4def-b236-918afa6df006 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:01.994264 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1075/2160] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:56:01 2022] POST /identity/v3/users => generated 301 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:01.998960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1086/2161] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:56:01.999100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35924c4-ea69-4c04-9067-1aa793882cf7 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:02.002207 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218baeb7-8e14-4def-b236-918afa6df006 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:02.012974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35924c4-ea69-4c04-9067-1aa793882cf7 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:02.016784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218baeb7-8e14-4def-b236-918afa6df006 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:02.020556 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe5e6f-2388-4cf2-bed2-f830ad2bc924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:02.021284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe5e6f-2388-4cf2-bed2-f830ad2bc924 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:02.021669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe5e6f-2388-4cf2-bed2-f830ad2bc924 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:02.021958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe5e6f-2388-4cf2-bed2-f830ad2bc924 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:02.022807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe5e6f-2388-4cf2-bed2-f830ad2bc924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:02.024452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe5e6f-2388-4cf2-bed2-f830ad2bc924 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:02.025212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bef95d-f9c6-497b-ab16-82ea6adc9349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:02.026142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bef95d-f9c6-497b-ab16-82ea6adc9349 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:02.026414 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bef95d-f9c6-497b-ab16-82ea6adc9349 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:02.027193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bef95d-f9c6-497b-ab16-82ea6adc9349 admin admin] PATH_INFO: `/v3/projects/865e2960151f475ea55b420d6056542f/users/a7af44d49c91488aa77eb7a3a32a8e69/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:02.035804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1076/2162] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:02.039723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe5e6f-2388-4cf2-bed2-f830ad2bc924 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:02.044562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bef95d-f9c6-497b-ab16-82ea6adc9349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=865e2960151f475ea55b420d6056542f, user_id=a7af44d49c91488aa77eb7a3a32a8e69, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:02.047590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bef95d-f9c6-497b-ab16-82ea6adc9349 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:02.051503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe5e6f-2388-4cf2-bed2-f830ad2bc924 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:02.057302 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a8909-3e7d-4105-be3e-bf90ddcf4bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:02.058222 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a8909-3e7d-4105-be3e-bf90ddcf4bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:02.058501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a8909-3e7d-4105-be3e-bf90ddcf4bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:02.058761 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a8909-3e7d-4105-be3e-bf90ddcf4bbf admin admin] PATH_INFO: `/v3/projects/865e24d366aa417e8271831e1129e96e/users/7fa9103e5fbf4393a1f3c9723daa6103/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:02.063342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1087/2163] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:02 2022] PUT /identity/v3/projects/865e2960151f475ea55b420d6056542f/users/a7af44d49c91488aa77eb7a3a32a8e69/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:02.073964 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9e96-6c81-4296-b77a-55b98e844f26 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:02.074332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9e96-6c81-4296-b77a-55b98e844f26 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:02.074626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9e96-6c81-4296-b77a-55b98e844f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:02.081976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a8909-3e7d-4105-be3e-bf90ddcf4bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=865e24d366aa417e8271831e1129e96e, user_id=7fa9103e5fbf4393a1f3c9723daa6103, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:02.085293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a8909-3e7d-4105-be3e-bf90ddcf4bbf admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:02.101724 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1077/2164] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:02 2022] PUT /identity/v3/projects/865e24d366aa417e8271831e1129e96e/users/7fa9103e5fbf4393a1f3c9723daa6103/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:56:02.106326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a8909-3e7d-4105-be3e-bf90ddcf4bbf admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:02.124752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-652e9e96-6c81-4296-b77a-55b98e844f26 None None] MFA Rules not processed for user `a7af44d49c91488aa77eb7a3a32a8e69`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:02.126529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-652e9e96-6c81-4296-b77a-55b98e844f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:02.174128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:02.178119 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:02.211606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a8909-3e7d-4105-be3e-bf90ddcf4bbf admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:02.216545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00489e77-624c-4687-a431-c8c79e480d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:02.217749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00489e77-624c-4687-a431-c8c79e480d60 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:02.218034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00489e77-624c-4687-a431-c8c79e480d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:02.218314 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00489e77-624c-4687-a431-c8c79e480d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:02.219124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00489e77-624c-4687-a431-c8c79e480d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:02.220214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1088/2165] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:56:02.220679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00489e77-624c-4687-a431-c8c79e480d60 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:02.231331 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9e96-6c81-4296-b77a-55b98e844f26 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:02.233260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-652e9e96-6c81-4296-b77a-55b98e844f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:02.236686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1078/2166] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:02.240625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00489e77-624c-4687-a431-c8c79e480d60 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:02.250679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00489e77-624c-4687-a431-c8c79e480d60 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:02.255676 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196c491-b8e8-44f4-84dc-d4697c3191d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:02.256559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196c491-b8e8-44f4-84dc-d4697c3191d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:02.256827 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196c491-b8e8-44f4-84dc-d4697c3191d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:02.257098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196c491-b8e8-44f4-84dc-d4697c3191d0 admin admin] PATH_INFO: `/v3/projects/865e24d366aa417e8271831e1129e96e/users/7fa9103e5fbf4393a1f3c9723daa6103/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:02.259314 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196c491-b8e8-44f4-84dc-d4697c3191d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=865e24d366aa417e8271831e1129e96e, user_id=7fa9103e5fbf4393a1f3c9723daa6103, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:02.262099 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196c491-b8e8-44f4-84dc-d4697c3191d0 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:02.273716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1079/2167] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:02 2022] PUT /identity/v3/projects/865e24d366aa417e8271831e1129e96e/users/7fa9103e5fbf4393a1f3c9723daa6103/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:02.288279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1ac30-47ec-448a-9048-313cfe2dc74a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:02.288279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1ac30-47ec-448a-9048-313cfe2dc74a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:02.288279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1ac30-47ec-448a-9048-313cfe2dc74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:02.292879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9e96-6c81-4296-b77a-55b98e844f26 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:02.313957 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-b1d1ac30-47ec-448a-9048-313cfe2dc74a None None] MFA Rules not processed for user `7fa9103e5fbf4393a1f3c9723daa6103`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:02.315273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b1d1ac30-47ec-448a-9048-313cfe2dc74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:02.345420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa44ad-8196-4f02-a8f8-931637e394d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:02.346074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa44ad-8196-4f02-a8f8-931637e394d5 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:02.346354 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa44ad-8196-4f02-a8f8-931637e394d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:02.346662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa44ad-8196-4f02-a8f8-931637e394d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:02.389544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1080/2168] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:56:02.404830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1ac30-47ec-448a-9048-313cfe2dc74a None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:02.406382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b1d1ac30-47ec-448a-9048-313cfe2dc74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:02.417386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa44ad-8196-4f02-a8f8-931637e394d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:02.418931 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1ac30-47ec-448a-9048-313cfe2dc74a None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:02.419083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa44ad-8196-4f02-a8f8-931637e394d5 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:02.422840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fee29-6c2b-4a43-9086-f4ef9572645a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:02.423519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fee29-6c2b-4a43-9086-f4ef9572645a service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:02.423835 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fee29-6c2b-4a43-9086-f4ef9572645a service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:02.424114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fee29-6c2b-4a43-9086-f4ef9572645a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:02.430576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1089/2169] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:02.439696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fee29-6c2b-4a43-9086-f4ef9572645a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:02.441198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fee29-6c2b-4a43-9086-f4ef9572645a service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:02.452039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1081/2170] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:56:02.838663 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa44ad-8196-4f02-a8f8-931637e394d5 service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:02.892067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:02.908323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa44ad-8196-4f02-a8f8-931637e394d5 service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:02.914602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:02.916316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:02.916316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:02.916316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] PATH_INFO: `/v3/users/7ba5493652a64aeb942c3c9c9f68d09b` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:02.917717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba5493652a64aeb942c3c9c9f68d09b)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:02.920367 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:02.988413 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba5493652a64aeb942c3c9c9f68d09b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.000103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba5493652a64aeb942c3c9c9f68d09b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.009160 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba5493652a64aeb942c3c9c9f68d09b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.009882 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba5493652a64aeb942c3c9c9f68d09b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.010551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba5493652a64aeb942c3c9c9f68d09b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.034853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1090/2171] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:02 2022] DELETE /identity/v3/users/7ba5493652a64aeb942c3c9c9f68d09b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:03.039450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29e1a0-eb65-4635-a090-ebbec58569c5 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:03.040199 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29e1a0-eb65-4635-a090-ebbec58569c5 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:03.040199 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29e1a0-eb65-4635-a090-ebbec58569c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:03.064463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-dc29e1a0-eb65-4635-a090-ebbec58569c5 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:03.065774 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-dc29e1a0-eb65-4635-a090-ebbec58569c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:03.155074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1082/2172] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:03.165326 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:03.167071 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:03.231614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec6b4a-e819-41ef-adb5-e0f54f21cfa2 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:03.237002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee9ba0-d1f0-48a4-8776-31292f83e43a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:03.237701 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee9ba0-d1f0-48a4-8776-31292f83e43a service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:03.237983 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee9ba0-d1f0-48a4-8776-31292f83e43a service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:03.238285 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee9ba0-d1f0-48a4-8776-31292f83e43a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:03.248309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee9ba0-d1f0-48a4-8776-31292f83e43a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:03.249733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee9ba0-d1f0-48a4-8776-31292f83e43a service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:03.260069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1091/2173] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:03.536473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29e1a0-eb65-4635-a090-ebbec58569c5 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:03.537944 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-dc29e1a0-eb65-4635-a090-ebbec58569c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:03.553889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29e1a0-eb65-4635-a090-ebbec58569c5 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:03.559059 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:03.559984 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:03.561105 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:03.561454 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] PATH_INFO: `/v3/projects/74eb269b1a194c83ab87a80ebbfb8c86` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:03.562644 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74eb269b1a194c83ab87a80ebbfb8c86)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:03.564711 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:03.596409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:03.614832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74eb269b1a194c83ab87a80ebbfb8c86', 'request_id': 'req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.621173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74eb269b1a194c83ab87a80ebbfb8c86', 'request_id': 'req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.630221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74eb269b1a194c83ab87a80ebbfb8c86', 'request_id': 'req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.641888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] The token cache is being invalidate because project 74eb269b1a194c83ab87a80ebbfb8c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:03.642173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.642816 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.644388 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1083/2174] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:03 2022] DELETE /identity/v3/projects/74eb269b1a194c83ab87a80ebbfb8c86 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:03.648325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee9ba0-d1f0-48a4-8776-31292f83e43a service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:03.658029 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee9ba0-d1f0-48a4-8776-31292f83e43a service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:03.715863 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b48d8d-0677-40df-a2f8-48b29fbb6e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:03.716873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b48d8d-0677-40df-a2f8-48b29fbb6e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:03.717227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b48d8d-0677-40df-a2f8-48b29fbb6e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:03.717508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b48d8d-0677-40df-a2f8-48b29fbb6e55 admin admin] PATH_INFO: `/v3/users/7fa9103e5fbf4393a1f3c9723daa6103` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:03.718732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b48d8d-0677-40df-a2f8-48b29fbb6e55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa9103e5fbf4393a1f3c9723daa6103)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:03.720773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b48d8d-0677-40df-a2f8-48b29fbb6e55 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:03.731156 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:03.776860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d7b48d8d-0677-40df-a2f8-48b29fbb6e55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa9103e5fbf4393a1f3c9723daa6103'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.785462 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d7b48d8d-0677-40df-a2f8-48b29fbb6e55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa9103e5fbf4393a1f3c9723daa6103'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.794056 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d7b48d8d-0677-40df-a2f8-48b29fbb6e55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa9103e5fbf4393a1f3c9723daa6103'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.794574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d7b48d8d-0677-40df-a2f8-48b29fbb6e55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa9103e5fbf4393a1f3c9723daa6103'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.795951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-d7b48d8d-0677-40df-a2f8-48b29fbb6e55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa9103e5fbf4393a1f3c9723daa6103'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:03.823621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1092/2175] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:03 2022] DELETE /identity/v3/users/7fa9103e5fbf4393a1f3c9723daa6103 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:04.148935 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:04.150408 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:04.179975 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba03ad2-20a7-485b-8a2f-56ea688e0fc8 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:04.244657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:04.245528 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:04.245808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:04.246239 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a admin admin] PATH_INFO: `/v3/projects/865e24d366aa417e8271831e1129e96e` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:04.247462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a admin admin] RBAC: Authorizing `identity:delete_project(project_id=865e24d366aa417e8271831e1129e96e)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:04.249647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:04.311756 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '865e24d366aa417e8271831e1129e96e', 'request_id': 'req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:04.318091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '865e24d366aa417e8271831e1129e96e', 'request_id': 'req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:04.325268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '865e24d366aa417e8271831e1129e96e', 'request_id': 'req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:04.336582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a admin admin] The token cache is being invalidate because project 865e24d366aa417e8271831e1129e96e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:04.336871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:04.337507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-58b01fc5-635a-4e3a-ad23-3abd6d6de01a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:04.339762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1084/2176] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:04 2022] DELETE /identity/v3/projects/865e24d366aa417e8271831e1129e96e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:04.344723 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc3a83-7bd3-4204-abd7-9b12d7521b11 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:04.345155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc3a83-7bd3-4204-abd7-9b12d7521b11 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:04.345538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc3a83-7bd3-4204-abd7-9b12d7521b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:04.368412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-4ddc3a83-7bd3-4204-abd7-9b12d7521b11 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:04.369650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4ddc3a83-7bd3-4204-abd7-9b12d7521b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:04.438419 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1093/2177] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:04.447353 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224270d2-c3dc-4917-a98f-a36e1254bc66 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:04.447639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224270d2-c3dc-4917-a98f-a36e1254bc66 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:04.447931 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224270d2-c3dc-4917-a98f-a36e1254bc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:04.467417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-224270d2-c3dc-4917-a98f-a36e1254bc66 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:04.468632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-224270d2-c3dc-4917-a98f-a36e1254bc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:04.476516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1085/2178] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:56:04.480770 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc3a83-7bd3-4204-abd7-9b12d7521b11 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:04.490772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc3a83-7bd3-4204-abd7-9b12d7521b11 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:04.495335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d330feb-c246-4919-9269-985158395ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:04.495976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d330feb-c246-4919-9269-985158395ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:04.496257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d330feb-c246-4919-9269-985158395ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:04.496534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d330feb-c246-4919-9269-985158395ce9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:04.497251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d330feb-c246-4919-9269-985158395ce9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:56:04.497558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d330feb-c246-4919-9269-985158395ce9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:04.499738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d330feb-c246-4919-9269-985158395ce9 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:04.511807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1094/2179] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:56:04.516618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224270d2-c3dc-4917-a98f-a36e1254bc66 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:04.525848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224270d2-c3dc-4917-a98f-a36e1254bc66 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:04.532736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfbc43-2e49-44f9-8d0a-7bb99c6b071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:04.532927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfbc43-2e49-44f9-8d0a-7bb99c6b071c admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:04.533101 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfbc43-2e49-44f9-8d0a-7bb99c6b071c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:04.533379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfbc43-2e49-44f9-8d0a-7bb99c6b071c admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:04.534398 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfbc43-2e49-44f9-8d0a-7bb99c6b071c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:04.536850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfbc43-2e49-44f9-8d0a-7bb99c6b071c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:04.563703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1086/2180] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:56:04 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:04.567845 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d330feb-c246-4919-9269-985158395ce9 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:04.578237 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d330feb-c246-4919-9269-985158395ce9 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:04.635979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b558f-ccf5-4794-8007-dd9f05ec4970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:04.636625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b558f-ccf5-4794-8007-dd9f05ec4970 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:04.636911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b558f-ccf5-4794-8007-dd9f05ec4970 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:04.637190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b558f-ccf5-4794-8007-dd9f05ec4970 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:04.637885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b558f-ccf5-4794-8007-dd9f05ec4970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:04.641391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b558f-ccf5-4794-8007-dd9f05ec4970 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:04.665335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1095/2181] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:56:04 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:04.668020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfbc43-2e49-44f9-8d0a-7bb99c6b071c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:04.678426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfbc43-2e49-44f9-8d0a-7bb99c6b071c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:04.683267 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881c6e3-1cab-4c3c-bf61-abf680ee176d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:04.683921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881c6e3-1cab-4c3c-bf61-abf680ee176d admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:04.684203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881c6e3-1cab-4c3c-bf61-abf680ee176d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:04.684475 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881c6e3-1cab-4c3c-bf61-abf680ee176d admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:04.685202 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881c6e3-1cab-4c3c-bf61-abf680ee176d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:04.687737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881c6e3-1cab-4c3c-bf61-abf680ee176d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:04.698770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1087/2182] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:04.703567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b558f-ccf5-4794-8007-dd9f05ec4970 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:04.710200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:04.713828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b558f-ccf5-4794-8007-dd9f05ec4970 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:04.721749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8ac04-6041-4661-8d00-2e0bae064347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:04.722445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8ac04-6041-4661-8d00-2e0bae064347 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:04.722965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8ac04-6041-4661-8d00-2e0bae064347 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:04.723274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8ac04-6041-4661-8d00-2e0bae064347 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:04.724190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8ac04-6041-4661-8d00-2e0bae064347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:04.725981 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8ac04-6041-4661-8d00-2e0bae064347 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:04.736730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1096/2183] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:04.741345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881c6e3-1cab-4c3c-bf61-abf680ee176d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:04.748862 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:04.752804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881c6e3-1cab-4c3c-bf61-abf680ee176d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:04.759229 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422bb66-8357-4a6b-8d6d-e38f1c569a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:04.760288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422bb66-8357-4a6b-8d6d-e38f1c569a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:04.760636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422bb66-8357-4a6b-8d6d-e38f1c569a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:04.761068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422bb66-8357-4a6b-8d6d-e38f1c569a2e admin admin] PATH_INFO: `/v3/projects/27f8860d53e34f2a976dc7abc28d000f/users/7bbc27e1d9204ffb833d982f67404e7e/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:04.776488 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422bb66-8357-4a6b-8d6d-e38f1c569a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f8860d53e34f2a976dc7abc28d000f, user_id=7bbc27e1d9204ffb833d982f67404e7e, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:04.779175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422bb66-8357-4a6b-8d6d-e38f1c569a2e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:04.792255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1088/2184] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:04 2022] PUT /identity/v3/projects/27f8860d53e34f2a976dc7abc28d000f/users/7bbc27e1d9204ffb833d982f67404e7e/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:04.803804 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02ce85-f103-4e28-87cf-67975c4808e7 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:04.804092 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02ce85-f103-4e28-87cf-67975c4808e7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:04.804374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02ce85-f103-4e28-87cf-67975c4808e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:04.835503 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ae02ce85-f103-4e28-87cf-67975c4808e7 None None] MFA Rules not processed for user `7bbc27e1d9204ffb833d982f67404e7e`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:04.837335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ae02ce85-f103-4e28-87cf-67975c4808e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:04.906291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1097/2185] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:04.917527 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422bb66-8357-4a6b-8d6d-e38f1c569a2e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:04.919285 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e422bb66-8357-4a6b-8d6d-e38f1c569a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:04.979109 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422bb66-8357-4a6b-8d6d-e38f1c569a2e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:04.983322 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5f323-f3f6-4084-910b-1e5b0546a125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:04.983953 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5f323-f3f6-4084-910b-1e5b0546a125 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:04.984232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5f323-f3f6-4084-910b-1e5b0546a125 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:04.984504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5f323-f3f6-4084-910b-1e5b0546a125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:04.992816 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5f323-f3f6-4084-910b-1e5b0546a125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:04.994275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5f323-f3f6-4084-910b-1e5b0546a125 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:05.005696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1089/2186] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:05.024951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02ce85-f103-4e28-87cf-67975c4808e7 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:05.103937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02ce85-f103-4e28-87cf-67975c4808e7 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:05.109103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:05.109993 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:05.110271 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:05.110541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] PATH_INFO: `/v3/users/7bbc27e1d9204ffb833d982f67404e7e` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:05.111767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bbc27e1d9204ffb833d982f67404e7e)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:05.113681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:05.173833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bbc27e1d9204ffb833d982f67404e7e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:05.180635 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bbc27e1d9204ffb833d982f67404e7e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:05.188461 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bbc27e1d9204ffb833d982f67404e7e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:05.189153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bbc27e1d9204ffb833d982f67404e7e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:05.189942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bbc27e1d9204ffb833d982f67404e7e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:05.209754 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1098/2187] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:05 2022] DELETE /identity/v3/users/7bbc27e1d9204ffb833d982f67404e7e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:05.214459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c98111-0a75-4761-9167-05750db3b20a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:05.214976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c98111-0a75-4761-9167-05750db3b20a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:05.215071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c98111-0a75-4761-9167-05750db3b20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:05.233411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a4c98111-0a75-4761-9167-05750db3b20a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:05.234619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a4c98111-0a75-4761-9167-05750db3b20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:05.295315 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1090/2188] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:05.305473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:05.306910 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:05.313178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:05.368338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db1ec7-4ac3-4bc6-96b5-43bd185f1ca2 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:05.372008 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3ff6d-6aa7-435a-b964-048cfc276f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:05.372655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3ff6d-6aa7-435a-b964-048cfc276f96 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:05.373004 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3ff6d-6aa7-435a-b964-048cfc276f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:05.373823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3ff6d-6aa7-435a-b964-048cfc276f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:05.385460 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3ff6d-6aa7-435a-b964-048cfc276f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:05.387149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3ff6d-6aa7-435a-b964-048cfc276f96 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:05.398170 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1099/2189] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:05.730121 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c98111-0a75-4761-9167-05750db3b20a None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:05.731681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a4c98111-0a75-4761-9167-05750db3b20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:05.746603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c98111-0a75-4761-9167-05750db3b20a None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:05.752559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:05.755620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:05.755620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:05.755620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd admin admin] PATH_INFO: `/v3/projects/27f8860d53e34f2a976dc7abc28d000f` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:05.756687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f8860d53e34f2a976dc7abc28d000f)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:05.758561 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:05.808023 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f8860d53e34f2a976dc7abc28d000f', 'request_id': 'req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:05.814716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f8860d53e34f2a976dc7abc28d000f', 'request_id': 'req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:05.822331 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f8860d53e34f2a976dc7abc28d000f', 'request_id': 'req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:05.832547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd admin admin] The token cache is being invalidate because project 27f8860d53e34f2a976dc7abc28d000f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:05.832836 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:05.833451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-15b99b6b-0ff2-4af2-b3aa-92acef3e19dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:05.835017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1091/2190] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:05 2022] DELETE /identity/v3/projects/27f8860d53e34f2a976dc7abc28d000f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:05.839948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f655ba0-f9a8-45d5-9e22-a57dfec83116 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:05.840410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f655ba0-f9a8-45d5-9e22-a57dfec83116 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:05.840503 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f655ba0-f9a8-45d5-9e22-a57dfec83116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:05.860698 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-5f655ba0-f9a8-45d5-9e22-a57dfec83116 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:05.861933 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5f655ba0-f9a8-45d5-9e22-a57dfec83116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:05.924849 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1100/2191] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:05.937579 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eab676-5cb2-4da2-8541-744255532d6a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:05.938011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eab676-5cb2-4da2-8541-744255532d6a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:05.938422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eab676-5cb2-4da2-8541-744255532d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:05.953288 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:05.959091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-15eab676-5cb2-4da2-8541-744255532d6a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:05.960885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-15eab676-5cb2-4da2-8541-744255532d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:05.969763 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1092/2192] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:05.974073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f655ba0-f9a8-45d5-9e22-a57dfec83116 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:05.985087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f655ba0-f9a8-45d5-9e22-a57dfec83116 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:05.989761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2280e1b-e65b-472f-b991-a3342b4c85b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:05.990415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2280e1b-e65b-472f-b991-a3342b4c85b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:05.990701 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2280e1b-e65b-472f-b991-a3342b4c85b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:05.991027 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2280e1b-e65b-472f-b991-a3342b4c85b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:05.991761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2280e1b-e65b-472f-b991-a3342b4c85b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:56:05.992067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2280e1b-e65b-472f-b991-a3342b4c85b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:05.993661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2280e1b-e65b-472f-b991-a3342b4c85b9 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:06.006947 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1101/2193] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:56:06.011157 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eab676-5cb2-4da2-8541-744255532d6a None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:06.020731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eab676-5cb2-4da2-8541-744255532d6a None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:06.025648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e8492-d83d-410f-be08-65a6b7c6ba06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:06.026312 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e8492-d83d-410f-be08-65a6b7c6ba06 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:06.026595 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e8492-d83d-410f-be08-65a6b7c6ba06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:06.026889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e8492-d83d-410f-be08-65a6b7c6ba06 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:06.027593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e8492-d83d-410f-be08-65a6b7c6ba06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:06.029620 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e8492-d83d-410f-be08-65a6b7c6ba06 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:06.056308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1093/2194] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:56:06 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:06.061168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2280e1b-e65b-472f-b991-a3342b4c85b9 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:06.071280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2280e1b-e65b-472f-b991-a3342b4c85b9 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:06.140584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55154124-d950-4f2c-b170-706ea76b931e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:06.141267 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55154124-d950-4f2c-b170-706ea76b931e admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:06.141550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55154124-d950-4f2c-b170-706ea76b931e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:06.141832 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55154124-d950-4f2c-b170-706ea76b931e admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:06.142543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55154124-d950-4f2c-b170-706ea76b931e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:06.144512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55154124-d950-4f2c-b170-706ea76b931e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:06.168782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1102/2195] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:56:06 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:06.171478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e8492-d83d-410f-be08-65a6b7c6ba06 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:06.181655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e8492-d83d-410f-be08-65a6b7c6ba06 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:06.186983 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b33294-b3aa-4492-8525-2dd7c0b185cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:06.187647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b33294-b3aa-4492-8525-2dd7c0b185cc admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:06.187952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b33294-b3aa-4492-8525-2dd7c0b185cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:06.188227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b33294-b3aa-4492-8525-2dd7c0b185cc admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:06.188979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b33294-b3aa-4492-8525-2dd7c0b185cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:06.190649 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b33294-b3aa-4492-8525-2dd7c0b185cc admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:06.202945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1094/2196] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:06.206965 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55154124-d950-4f2c-b170-706ea76b931e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:06.217624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55154124-d950-4f2c-b170-706ea76b931e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:06.222539 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c7dbd-01b8-4ffe-ad6a-d995d32b78db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:06.223220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c7dbd-01b8-4ffe-ad6a-d995d32b78db admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:06.223501 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c7dbd-01b8-4ffe-ad6a-d995d32b78db admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:06.223773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c7dbd-01b8-4ffe-ad6a-d995d32b78db admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:06.224490 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c7dbd-01b8-4ffe-ad6a-d995d32b78db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:06.226114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c7dbd-01b8-4ffe-ad6a-d995d32b78db admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:06.238233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1103/2197] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:06.242381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b33294-b3aa-4492-8525-2dd7c0b185cc admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:06.252557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b33294-b3aa-4492-8525-2dd7c0b185cc admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:06.257439 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bdcfe-ed01-4a09-977b-ed8155f7d2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:06.258338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bdcfe-ed01-4a09-977b-ed8155f7d2dd admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:06.258618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bdcfe-ed01-4a09-977b-ed8155f7d2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:06.258927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bdcfe-ed01-4a09-977b-ed8155f7d2dd admin admin] PATH_INFO: `/v3/projects/8f6f5136266e48fb815558fdd4a2b342/users/8a54157c9097448b98a9e3c0e77f3e02/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:06.275792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008bdcfe-ed01-4a09-977b-ed8155f7d2dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6f5136266e48fb815558fdd4a2b342, user_id=8a54157c9097448b98a9e3c0e77f3e02, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:06.278471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008bdcfe-ed01-4a09-977b-ed8155f7d2dd admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:06.294353 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1095/2198] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:06 2022] PUT /identity/v3/projects/8f6f5136266e48fb815558fdd4a2b342/users/8a54157c9097448b98a9e3c0e77f3e02/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:06.306532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b2662-44b0-434d-8b5c-c47594a74627 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:06.307020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b2662-44b0-434d-8b5c-c47594a74627 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:06.307277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b2662-44b0-434d-8b5c-c47594a74627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:06.326466 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:06.335026 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f68b2662-44b0-434d-8b5c-c47594a74627 None None] MFA Rules not processed for user `8a54157c9097448b98a9e3c0e77f3e02`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:06.336599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f68b2662-44b0-434d-8b5c-c47594a74627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:06.405015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1104/2199] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:06.416414 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bdcfe-ed01-4a09-977b-ed8155f7d2dd admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:06.418080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-008bdcfe-ed01-4a09-977b-ed8155f7d2dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:06.476584 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bdcfe-ed01-4a09-977b-ed8155f7d2dd admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:06.480300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ac44a-2a52-43e1-bc9b-8b530fe8c435 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:06.480947 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ac44a-2a52-43e1-bc9b-8b530fe8c435 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:06.481235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ac44a-2a52-43e1-bc9b-8b530fe8c435 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:06.481518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ac44a-2a52-43e1-bc9b-8b530fe8c435 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:06.490877 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ac44a-2a52-43e1-bc9b-8b530fe8c435 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:06.492687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ac44a-2a52-43e1-bc9b-8b530fe8c435 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:06.502663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1096/2200] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:06.530593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b2662-44b0-434d-8b5c-c47594a74627 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:06.598764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b2662-44b0-434d-8b5c-c47594a74627 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:06.606828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:06.607968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:06.608246 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:06.608541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] PATH_INFO: `/v3/users/8a54157c9097448b98a9e3c0e77f3e02` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:06.609789 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a54157c9097448b98a9e3c0e77f3e02)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:06.611821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:06.663680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a54157c9097448b98a9e3c0e77f3e02'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:06.681442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a54157c9097448b98a9e3c0e77f3e02'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:06.683752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a54157c9097448b98a9e3c0e77f3e02'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:06.684536 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a54157c9097448b98a9e3c0e77f3e02'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:06.685214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a54157c9097448b98a9e3c0e77f3e02'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:06.708066 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1105/2201] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:06 2022] DELETE /identity/v3/users/8a54157c9097448b98a9e3c0e77f3e02 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:06.712600 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc4f7e-1959-46e2-8e84-1984a573a695 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:06.712920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc4f7e-1959-46e2-8e84-1984a573a695 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:06.713204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc4f7e-1959-46e2-8e84-1984a573a695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:06.734637 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-5ebc4f7e-1959-46e2-8e84-1984a573a695 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:06.735887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5ebc4f7e-1959-46e2-8e84-1984a573a695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:06.805215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1097/2202] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:56:06.815639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:06.817068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:06.839994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:06.885898 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d281b3-6d7a-4afb-a35f-f9f0e9470ca6 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:06.889900 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334622c5-2777-4ef3-85be-93eaf69faf03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:06.890551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334622c5-2777-4ef3-85be-93eaf69faf03 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:06.890838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334622c5-2777-4ef3-85be-93eaf69faf03 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:06.891153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334622c5-2777-4ef3-85be-93eaf69faf03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:06.900495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334622c5-2777-4ef3-85be-93eaf69faf03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:06.901925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334622c5-2777-4ef3-85be-93eaf69faf03 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:06.911924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1106/2203] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:07.192721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc4f7e-1959-46e2-8e84-1984a573a695 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:07.194041 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5ebc4f7e-1959-46e2-8e84-1984a573a695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:07.208792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc4f7e-1959-46e2-8e84-1984a573a695 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:07.214094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:07.215074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:07.215361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:07.215635 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4 admin admin] PATH_INFO: `/v3/projects/8f6f5136266e48fb815558fdd4a2b342` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:07.220160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6f5136266e48fb815558fdd4a2b342)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:07.222102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:07.267820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:07.276789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6f5136266e48fb815558fdd4a2b342', 'request_id': 'req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:07.284027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6f5136266e48fb815558fdd4a2b342', 'request_id': 'req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:07.291014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6f5136266e48fb815558fdd4a2b342', 'request_id': 'req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:07.301408 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4 admin admin] The token cache is being invalidate because project 8f6f5136266e48fb815558fdd4a2b342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:07.301715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:07.302353 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0e5b8029-bca9-46a1-a41c-3e95d723f7d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:07.303944 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1098/2204] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:07 2022] DELETE /identity/v3/projects/8f6f5136266e48fb815558fdd4a2b342 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:07.308759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201337e-feb9-416a-9316-1a501c875379 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:07.309143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201337e-feb9-416a-9316-1a501c875379 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:07.309372 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201337e-feb9-416a-9316-1a501c875379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:07.328367 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-5201337e-feb9-416a-9316-1a501c875379 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:07.329572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5201337e-feb9-416a-9316-1a501c875379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:07.391923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1107/2205] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:07.399797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f89646-5b4c-4369-8fa7-3e52eb75fad0 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:07.400093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f89646-5b4c-4369-8fa7-3e52eb75fad0 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:07.400379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f89646-5b4c-4369-8fa7-3e52eb75fad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:07.419357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-07f89646-5b4c-4369-8fa7-3e52eb75fad0 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:07.420809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-07f89646-5b4c-4369-8fa7-3e52eb75fad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:07.428667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1099/2206] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:07.433003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201337e-feb9-416a-9316-1a501c875379 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:07.442788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201337e-feb9-416a-9316-1a501c875379 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:07.447575 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa869a-53f0-4ed6-bdd3-51218a3e29c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:07.448224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa869a-53f0-4ed6-bdd3-51218a3e29c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:07.448511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa869a-53f0-4ed6-bdd3-51218a3e29c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:07.448789 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa869a-53f0-4ed6-bdd3-51218a3e29c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:07.449515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa869a-53f0-4ed6-bdd3-51218a3e29c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:56:07.449832 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa869a-53f0-4ed6-bdd3-51218a3e29c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:07.452178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa869a-53f0-4ed6-bdd3-51218a3e29c7 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:07.463878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1108/2207] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:56:07.469072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f89646-5b4c-4369-8fa7-3e52eb75fad0 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:07.478803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f89646-5b4c-4369-8fa7-3e52eb75fad0 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:07.484580 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2edc87-95e0-48fc-8f1b-b16867e7a3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:07.485242 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edc87-95e0-48fc-8f1b-b16867e7a3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:07.485529 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edc87-95e0-48fc-8f1b-b16867e7a3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:07.485811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edc87-95e0-48fc-8f1b-b16867e7a3b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:07.486503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2edc87-95e0-48fc-8f1b-b16867e7a3b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:07.488456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2edc87-95e0-48fc-8f1b-b16867e7a3b6 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:07.514730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1100/2208] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:56:07 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:07.519256 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa869a-53f0-4ed6-bdd3-51218a3e29c7 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:07.530590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa869a-53f0-4ed6-bdd3-51218a3e29c7 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:07.589251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e999c6-436a-4465-abf8-39cccf321b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:07.589952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e999c6-436a-4465-abf8-39cccf321b9f admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:07.590234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e999c6-436a-4465-abf8-39cccf321b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:07.590510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e999c6-436a-4465-abf8-39cccf321b9f admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:07.591248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e999c6-436a-4465-abf8-39cccf321b9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:07.594792 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e999c6-436a-4465-abf8-39cccf321b9f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:07.619072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1109/2209] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:56:07 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:07.620329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2edc87-95e0-48fc-8f1b-b16867e7a3b6 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:07.631647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2edc87-95e0-48fc-8f1b-b16867e7a3b6 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:07.636501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddadec-cc0f-43db-8c2f-ec0867e237c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:07.637241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddadec-cc0f-43db-8c2f-ec0867e237c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:07.637521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddadec-cc0f-43db-8c2f-ec0867e237c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:07.637795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddadec-cc0f-43db-8c2f-ec0867e237c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:07.639163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eddadec-cc0f-43db-8c2f-ec0867e237c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:07.640811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eddadec-cc0f-43db-8c2f-ec0867e237c4 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:07.651812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1101/2210] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:07.655848 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e999c6-436a-4465-abf8-39cccf321b9f admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:07.665778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e999c6-436a-4465-abf8-39cccf321b9f admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:07.671148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bd034-3664-4f68-b749-f2bb3306a71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:07.671794 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bd034-3664-4f68-b749-f2bb3306a71c admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:07.672074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bd034-3664-4f68-b749-f2bb3306a71c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:07.672347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bd034-3664-4f68-b749-f2bb3306a71c admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:07.673064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0bd034-3664-4f68-b749-f2bb3306a71c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:07.674627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0bd034-3664-4f68-b749-f2bb3306a71c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:07.685018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1110/2211] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:07.689005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddadec-cc0f-43db-8c2f-ec0867e237c4 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:07.699710 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddadec-cc0f-43db-8c2f-ec0867e237c4 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:07.705657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def4a23-245e-4ea0-b09e-2de3778e2eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:07.706626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def4a23-245e-4ea0-b09e-2de3778e2eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:07.707036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def4a23-245e-4ea0-b09e-2de3778e2eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:07.707338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def4a23-245e-4ea0-b09e-2de3778e2eaf admin admin] PATH_INFO: `/v3/projects/3640112bc8fc4cbebed43506d47a2306/users/bcd8dd0ff3f04d5eac4af09f36b625a3/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:07.727031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def4a23-245e-4ea0-b09e-2de3778e2eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3640112bc8fc4cbebed43506d47a2306, user_id=bcd8dd0ff3f04d5eac4af09f36b625a3, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:07.730713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def4a23-245e-4ea0-b09e-2de3778e2eaf admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:07.759041 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1102/2212] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:07 2022] PUT /identity/v3/projects/3640112bc8fc4cbebed43506d47a2306/users/bcd8dd0ff3f04d5eac4af09f36b625a3/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:07.770297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098c775-53b5-43a2-ad3a-3d2eb2b3a274 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:07.770608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098c775-53b5-43a2-ad3a-3d2eb2b3a274 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:07.770966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098c775-53b5-43a2-ad3a-3d2eb2b3a274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:07.801365 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-0098c775-53b5-43a2-ad3a-3d2eb2b3a274 None None] MFA Rules not processed for user `bcd8dd0ff3f04d5eac4af09f36b625a3`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:07.802797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-0098c775-53b5-43a2-ad3a-3d2eb2b3a274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:07.823293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:07.876073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1111/2213] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:07.888240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def4a23-245e-4ea0-b09e-2de3778e2eaf admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:07.890913 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3def4a23-245e-4ea0-b09e-2de3778e2eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:07.958300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def4a23-245e-4ea0-b09e-2de3778e2eaf admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:07.962403 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fcf09-3771-4910-907d-4cff7bd2a97d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:07.963189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fcf09-3771-4910-907d-4cff7bd2a97d service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:07.963484 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fcf09-3771-4910-907d-4cff7bd2a97d service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:07.963758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fcf09-3771-4910-907d-4cff7bd2a97d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:07.978478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fcf09-3771-4910-907d-4cff7bd2a97d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:07.980022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fcf09-3771-4910-907d-4cff7bd2a97d service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:07.991510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1103/2214] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:13.197854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098c775-53b5-43a2-ad3a-3d2eb2b3a274 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:13.199443 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-0098c775-53b5-43a2-ad3a-3d2eb2b3a274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:13.340196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098c775-53b5-43a2-ad3a-3d2eb2b3a274 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:13.344775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:13.345662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:13.345943 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:13.346216 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] PATH_INFO: `/v3/users/a7af44d49c91488aa77eb7a3a32a8e69` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:13.347425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7af44d49c91488aa77eb7a3a32a8e69)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:13.349348 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:13.398479 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7af44d49c91488aa77eb7a3a32a8e69'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:13.406385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7af44d49c91488aa77eb7a3a32a8e69'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:13.411183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:13.415428 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7af44d49c91488aa77eb7a3a32a8e69'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:13.416341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7af44d49c91488aa77eb7a3a32a8e69'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:13.417312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7af44d49c91488aa77eb7a3a32a8e69'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:13.442033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1112/2215] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:13 2022] DELETE /identity/v3/users/a7af44d49c91488aa77eb7a3a32a8e69 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:13.447085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414eacde-343b-48a8-a746-f32ac3334b50 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:13.447629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414eacde-343b-48a8-a746-f32ac3334b50 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:13.447720 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414eacde-343b-48a8-a746-f32ac3334b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:13.475006 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-414eacde-343b-48a8-a746-f32ac3334b50 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:13.476216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-414eacde-343b-48a8-a746-f32ac3334b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:13.536008 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1104/2216] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:13.547989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:13.549962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:13.658880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c787d6-9b6a-4c51-a63e-747d57b5c622 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:13.662783 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946fb25-af81-4124-b4e4-b031457f8cf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:13.663471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946fb25-af81-4124-b4e4-b031457f8cf1 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:13.663762 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946fb25-af81-4124-b4e4-b031457f8cf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:13.664035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946fb25-af81-4124-b4e4-b031457f8cf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:13.674239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946fb25-af81-4124-b4e4-b031457f8cf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:13.675727 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946fb25-af81-4124-b4e4-b031457f8cf1 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:13.686045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1113/2217] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:13.999916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414eacde-343b-48a8-a746-f32ac3334b50 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:14.001983 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-414eacde-343b-48a8-a746-f32ac3334b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:14.010444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:14.016992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414eacde-343b-48a8-a746-f32ac3334b50 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:14.022524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:14.024251 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:14.024417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:14.024738 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] PATH_INFO: `/v3/projects/865e2960151f475ea55b420d6056542f` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:14.026324 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=865e2960151f475ea55b420d6056542f)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:14.030354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:14.079371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '865e2960151f475ea55b420d6056542f', 'request_id': 'req-3b19dde1-7a7c-4611-a6ad-fac503515ad4'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:14.085349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '865e2960151f475ea55b420d6056542f', 'request_id': 'req-3b19dde1-7a7c-4611-a6ad-fac503515ad4'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:14.092914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '865e2960151f475ea55b420d6056542f', 'request_id': 'req-3b19dde1-7a7c-4611-a6ad-fac503515ad4'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:14.104234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] The token cache is being invalidate because project 865e2960151f475ea55b420d6056542f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:14.104521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:14.105164 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:14.106781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1105/2218] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:13 2022] DELETE /identity/v3/projects/865e2960151f475ea55b420d6056542f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:23.681722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=51,keystone:DELETE=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:24.100961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:25.505553 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff29de4-cfa8-49ca-85a6-8ff28af21953 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:25.506032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff29de4-cfa8-49ca-85a6-8ff28af21953 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:25.506129 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff29de4-cfa8-49ca-85a6-8ff28af21953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:25.537176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-3ff29de4-cfa8-49ca-85a6-8ff28af21953 None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:25.538516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3ff29de4-cfa8-49ca-85a6-8ff28af21953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:25.600340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1114/2219] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 13:56:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:25.647916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:25.649501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:25.725592 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19dde1-7a7c-4611-a6ad-fac503515ad4 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:25.730590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ab180-3b31-4a5c-a0d6-69ab14aa4173 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:25.731279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ab180-3b31-4a5c-a0d6-69ab14aa4173 service swift] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:25.731568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ab180-3b31-4a5c-a0d6-69ab14aa4173 service swift] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:25.731846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ab180-3b31-4a5c-a0d6-69ab14aa4173 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:25.740249 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ab180-3b31-4a5c-a0d6-69ab14aa4173 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:25.741834 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ab180-3b31-4a5c-a0d6-69ab14aa4173 service swift] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:25.749905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1106/2220] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 13:56:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:56:29.638392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff29de4-cfa8-49ca-85a6-8ff28af21953 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:29.640118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3ff29de4-cfa8-49ca-85a6-8ff28af21953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:29.750888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff29de4-cfa8-49ca-85a6-8ff28af21953 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:29.756510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:29.757383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:29.757662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:29.757943 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] PATH_INFO: `/v3/users/bcd8dd0ff3f04d5eac4af09f36b625a3` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:29.759591 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd8dd0ff3f04d5eac4af09f36b625a3)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:29.761523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:29.821396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd8dd0ff3f04d5eac4af09f36b625a3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:29.832002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd8dd0ff3f04d5eac4af09f36b625a3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:29.846647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd8dd0ff3f04d5eac4af09f36b625a3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:29.847797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd8dd0ff3f04d5eac4af09f36b625a3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:29.848428 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd8dd0ff3f04d5eac4af09f36b625a3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:29.878784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1115/2221] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:29 2022] DELETE /identity/v3/users/bcd8dd0ff3f04d5eac4af09f36b625a3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:29.881311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641f440-6905-4d80-9ffa-5d14cf4caac6 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:29.882252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641f440-6905-4d80-9ffa-5d14cf4caac6 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:29.882694 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641f440-6905-4d80-9ffa-5d14cf4caac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:29.904498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-e641f440-6905-4d80-9ffa-5d14cf4caac6 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:29.905732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e641f440-6905-4d80-9ffa-5d14cf4caac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:29.995000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1107/2222] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:30.005592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:30.007115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:30.035939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:30.087277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6f9b3-f746-4a2b-8d93-51800c7aad55 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:30.092183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346882e6-a267-451e-af62-f4f5d92663be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:30.092750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346882e6-a267-451e-af62-f4f5d92663be service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:30.093038 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346882e6-a267-451e-af62-f4f5d92663be service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:30.093320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346882e6-a267-451e-af62-f4f5d92663be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:30.104923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346882e6-a267-451e-af62-f4f5d92663be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:30.106432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346882e6-a267-451e-af62-f4f5d92663be service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:30.131085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1116/2223] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:30.455456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641f440-6905-4d80-9ffa-5d14cf4caac6 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:30.457077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e641f440-6905-4d80-9ffa-5d14cf4caac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:30.478934 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641f440-6905-4d80-9ffa-5d14cf4caac6 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:30.490705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:30.491622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:30.491909 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:30.492248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a admin admin] PATH_INFO: `/v3/projects/3640112bc8fc4cbebed43506d47a2306` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:30.493405 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3640112bc8fc4cbebed43506d47a2306)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:30.495343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:30.556681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3640112bc8fc4cbebed43506d47a2306', 'request_id': 'req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:30.571767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3640112bc8fc4cbebed43506d47a2306', 'request_id': 'req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:30.574341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3640112bc8fc4cbebed43506d47a2306', 'request_id': 'req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:30.588814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a admin admin] The token cache is being invalidate because project 3640112bc8fc4cbebed43506d47a2306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:30.589122 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:30.589819 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8d4883bd-8b59-4824-8fa5-44923f5b0d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:30.591842 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1108/2224] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:30 2022] DELETE /identity/v3/projects/3640112bc8fc4cbebed43506d47a2306 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:30.598956 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb2dfb-ebff-4ba4-affc-18408c3fcac0 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:30.599301 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb2dfb-ebff-4ba4-affc-18408c3fcac0 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:30.599596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb2dfb-ebff-4ba4-affc-18408c3fcac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:30.655572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-58eb2dfb-ebff-4ba4-affc-18408c3fcac0 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:30.656864 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-58eb2dfb-ebff-4ba4-affc-18408c3fcac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:30.733114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1117/2225] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:30.734893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837964e-0318-43e8-911d-577e3ee0f271 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:30.735823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837964e-0318-43e8-911d-577e3ee0f271 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:30.736247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837964e-0318-43e8-911d-577e3ee0f271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:30.757109 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-f837964e-0318-43e8-911d-577e3ee0f271 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:30.758482 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f837964e-0318-43e8-911d-577e3ee0f271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:30.767435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1109/2226] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:30.772204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb2dfb-ebff-4ba4-affc-18408c3fcac0 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:30.786744 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb2dfb-ebff-4ba4-affc-18408c3fcac0 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:30.793791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfacb1-41aa-4a76-9d55-9824fdfcf76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:30.794516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfacb1-41aa-4a76-9d55-9824fdfcf76c admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:30.794804 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfacb1-41aa-4a76-9d55-9824fdfcf76c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:30.795148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfacb1-41aa-4a76-9d55-9824fdfcf76c admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:30.795934 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfacb1-41aa-4a76-9d55-9824fdfcf76c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:56:30.796252 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfacb1-41aa-4a76-9d55-9824fdfcf76c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:30.799527 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfacb1-41aa-4a76-9d55-9824fdfcf76c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:30.814308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1118/2227] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:56:30.818815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837964e-0318-43e8-911d-577e3ee0f271 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:30.828601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837964e-0318-43e8-911d-577e3ee0f271 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:30.834022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dba271-e42e-4512-98bf-c0351c0e6b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:30.834690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dba271-e42e-4512-98bf-c0351c0e6b63 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:30.835014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dba271-e42e-4512-98bf-c0351c0e6b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:30.835299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dba271-e42e-4512-98bf-c0351c0e6b63 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:30.835988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dba271-e42e-4512-98bf-c0351c0e6b63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:30.837885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dba271-e42e-4512-98bf-c0351c0e6b63 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:30.841721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:30.871871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1110/2228] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:56:30 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:30.876255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfacb1-41aa-4a76-9d55-9824fdfcf76c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:30.888123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfacb1-41aa-4a76-9d55-9824fdfcf76c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:30.956131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1189e-2252-45e3-8433-eb0c5600b925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:30.957885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1189e-2252-45e3-8433-eb0c5600b925 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:30.958338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1189e-2252-45e3-8433-eb0c5600b925 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:30.958710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1189e-2252-45e3-8433-eb0c5600b925 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:30.960495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1189e-2252-45e3-8433-eb0c5600b925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:30.963555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1189e-2252-45e3-8433-eb0c5600b925 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:30.993176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1119/2229] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:56:30 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:30.994421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dba271-e42e-4512-98bf-c0351c0e6b63 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:31.007351 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dba271-e42e-4512-98bf-c0351c0e6b63 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:31.013040 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a02b80-9900-469a-b1ac-9e76276f951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:31.013838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a02b80-9900-469a-b1ac-9e76276f951a admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:31.014258 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a02b80-9900-469a-b1ac-9e76276f951a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:31.014663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a02b80-9900-469a-b1ac-9e76276f951a admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:31.015603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a02b80-9900-469a-b1ac-9e76276f951a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:31.017703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a02b80-9900-469a-b1ac-9e76276f951a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:31.030963 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1111/2230] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:56:31.034882 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1189e-2252-45e3-8433-eb0c5600b925 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:31.046134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1189e-2252-45e3-8433-eb0c5600b925 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:31.051700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f68c72-b295-45db-b73d-d858d972bdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:31.052528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f68c72-b295-45db-b73d-d858d972bdb8 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:31.052820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f68c72-b295-45db-b73d-d858d972bdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:31.053101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f68c72-b295-45db-b73d-d858d972bdb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:31.064103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f68c72-b295-45db-b73d-d858d972bdb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:31.066474 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f68c72-b295-45db-b73d-d858d972bdb8 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:31.074966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:31.085012 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1120/2231] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:31.089256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a02b80-9900-469a-b1ac-9e76276f951a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:31.100510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a02b80-9900-469a-b1ac-9e76276f951a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:31.106230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa257b38-14c7-42ac-9955-98f4f4ef9553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:31.107393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa257b38-14c7-42ac-9955-98f4f4ef9553 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:31.107796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa257b38-14c7-42ac-9955-98f4f4ef9553 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:31.108171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa257b38-14c7-42ac-9955-98f4f4ef9553 admin admin] PATH_INFO: `/v3/projects/a51cbae6cb494a2fac656e416fe868c2/users/a1274435c6774faa9eeeac6726b52b47/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:31.134140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa257b38-14c7-42ac-9955-98f4f4ef9553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a51cbae6cb494a2fac656e416fe868c2, user_id=a1274435c6774faa9eeeac6726b52b47, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:31.136928 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa257b38-14c7-42ac-9955-98f4f4ef9553 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:31.156709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1112/2232] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:31 2022] PUT /identity/v3/projects/a51cbae6cb494a2fac656e416fe868c2/users/a1274435c6774faa9eeeac6726b52b47/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:31.162406 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cebc43-68a3-4fed-b77c-2da9c57a288d None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:31.162916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cebc43-68a3-4fed-b77c-2da9c57a288d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:31.163339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cebc43-68a3-4fed-b77c-2da9c57a288d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:31.183843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-a4cebc43-68a3-4fed-b77c-2da9c57a288d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:31.185173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a4cebc43-68a3-4fed-b77c-2da9c57a288d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:31.268225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1121/2233] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:31.284573 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa257b38-14c7-42ac-9955-98f4f4ef9553 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:31.286669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-fa257b38-14c7-42ac-9955-98f4f4ef9553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:31.369433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa257b38-14c7-42ac-9955-98f4f4ef9553 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:31.375167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb429ba7-8379-442b-b936-4a0c78e959fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:31.375851 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb429ba7-8379-442b-b936-4a0c78e959fc service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:31.376813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb429ba7-8379-442b-b936-4a0c78e959fc service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:31.376813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb429ba7-8379-442b-b936-4a0c78e959fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:31.387671 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb429ba7-8379-442b-b936-4a0c78e959fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:31.390896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb429ba7-8379-442b-b936-4a0c78e959fc service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:31.408008 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1113/2234] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:34.648838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cebc43-68a3-4fed-b77c-2da9c57a288d None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:34.650743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a4cebc43-68a3-4fed-b77c-2da9c57a288d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:34.670267 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cebc43-68a3-4fed-b77c-2da9c57a288d None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:34.696365 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:34.697519 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:34.697805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:34.698083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] PATH_INFO: `/v3/users/1b630eac3ac2443d8cb54a7901ca88e7` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:34.699566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b630eac3ac2443d8cb54a7901ca88e7)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:34.701506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:34.815037 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b630eac3ac2443d8cb54a7901ca88e7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:34.825933 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b630eac3ac2443d8cb54a7901ca88e7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:34.852737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b630eac3ac2443d8cb54a7901ca88e7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:34.852891 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b630eac3ac2443d8cb54a7901ca88e7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:34.853444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b630eac3ac2443d8cb54a7901ca88e7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:34.884959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1122/2235] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:34 2022] DELETE /identity/v3/users/1b630eac3ac2443d8cb54a7901ca88e7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:34.890042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44e88f-915f-4652-8038-403c633a1c7a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:34.890497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44e88f-915f-4652-8038-403c633a1c7a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:34.890600 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44e88f-915f-4652-8038-403c633a1c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:34.950094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-2a44e88f-915f-4652-8038-403c633a1c7a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:34.953670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2a44e88f-915f-4652-8038-403c633a1c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:35.116257 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1114/2236] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:35.151647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:35.153027 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:35.249197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a5500-3912-4568-97bd-dd0ff57a207f admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:35.254628 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcbad4-d396-4382-9609-984f096d9f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:35.255312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcbad4-d396-4382-9609-984f096d9f84 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:35.255592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcbad4-d396-4382-9609-984f096d9f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:35.255867 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcbad4-d396-4382-9609-984f096d9f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:35.266948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcbad4-d396-4382-9609-984f096d9f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:35.268491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcbad4-d396-4382-9609-984f096d9f84 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:35.281460 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1123/2237] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:36.019785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44e88f-915f-4652-8038-403c633a1c7a None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:36.022655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2a44e88f-915f-4652-8038-403c633a1c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:36.066913 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44e88f-915f-4652-8038-403c633a1c7a None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:36.077608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dd001-5fa1-416d-85b8-4e48f2c473de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:36.079643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dd001-5fa1-416d-85b8-4e48f2c473de admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:36.080106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dd001-5fa1-416d-85b8-4e48f2c473de admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:36.083820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dd001-5fa1-416d-85b8-4e48f2c473de admin admin] PATH_INFO: `/v3/projects/35444f2dc65a40828956a113763d3259` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:36.085862 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dd001-5fa1-416d-85b8-4e48f2c473de admin admin] RBAC: Authorizing `identity:delete_project(project_id=35444f2dc65a40828956a113763d3259)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:36.092092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dd001-5fa1-416d-85b8-4e48f2c473de admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:36.223003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:36.225874 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-de8dd001-5fa1-416d-85b8-4e48f2c473de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35444f2dc65a40828956a113763d3259', 'request_id': 'req-de8dd001-5fa1-416d-85b8-4e48f2c473de'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:36.236809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-de8dd001-5fa1-416d-85b8-4e48f2c473de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35444f2dc65a40828956a113763d3259', 'request_id': 'req-de8dd001-5fa1-416d-85b8-4e48f2c473de'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:36.249975 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-de8dd001-5fa1-416d-85b8-4e48f2c473de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35444f2dc65a40828956a113763d3259', 'request_id': 'req-de8dd001-5fa1-416d-85b8-4e48f2c473de'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:36.269000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-de8dd001-5fa1-416d-85b8-4e48f2c473de admin admin] The token cache is being invalidate because project 35444f2dc65a40828956a113763d3259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:36.269532 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-de8dd001-5fa1-416d-85b8-4e48f2c473de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:36.270487 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-de8dd001-5fa1-416d-85b8-4e48f2c473de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:36.272867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1115/2238] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:36 2022] DELETE /identity/v3/projects/35444f2dc65a40828956a113763d3259 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:36.278603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85915f89-e56c-473e-94d5-22459dedd94f None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:36.278603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85915f89-e56c-473e-94d5-22459dedd94f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:36.279274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85915f89-e56c-473e-94d5-22459dedd94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:36.308243 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-85915f89-e56c-473e-94d5-22459dedd94f None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:36.310026 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-85915f89-e56c-473e-94d5-22459dedd94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:36.322407 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:36.448685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1124/2239] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:36.457233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8249b7-0142-4ecf-a8f1-268a08f794b9 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:36.457386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8249b7-0142-4ecf-a8f1-268a08f794b9 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:36.457866 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8249b7-0142-4ecf-a8f1-268a08f794b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:36.486379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-9d8249b7-0142-4ecf-a8f1-268a08f794b9 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:36.487936 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9d8249b7-0142-4ecf-a8f1-268a08f794b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:36.497335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1116/2240] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:36.501392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85915f89-e56c-473e-94d5-22459dedd94f None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:36.517409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85915f89-e56c-473e-94d5-22459dedd94f None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:36.523371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acf048-4ae7-42b9-afff-d32b64427df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:36.524976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acf048-4ae7-42b9-afff-d32b64427df9 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:36.525349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acf048-4ae7-42b9-afff-d32b64427df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:36.525790 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acf048-4ae7-42b9-afff-d32b64427df9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:36.527176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92acf048-4ae7-42b9-afff-d32b64427df9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:56:36.527573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92acf048-4ae7-42b9-afff-d32b64427df9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:36.529402 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92acf048-4ae7-42b9-afff-d32b64427df9 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:36.548958 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1125/2241] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:56:36.552770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8249b7-0142-4ecf-a8f1-268a08f794b9 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:36.565172 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8249b7-0142-4ecf-a8f1-268a08f794b9 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:36.573955 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099749f-2373-4c3a-b096-0ce3634d247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:36.574843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099749f-2373-4c3a-b096-0ce3634d247f admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:36.575619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099749f-2373-4c3a-b096-0ce3634d247f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:36.576105 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099749f-2373-4c3a-b096-0ce3634d247f admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:36.577159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099749f-2373-4c3a-b096-0ce3634d247f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:36.580551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099749f-2373-4c3a-b096-0ce3634d247f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:36.626331 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1117/2242] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:56:36 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:36.628125 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acf048-4ae7-42b9-afff-d32b64427df9 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:36.648507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acf048-4ae7-42b9-afff-d32b64427df9 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:36.753088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c708c62-bbdf-4462-8122-abf96a67c662 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:36.753576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c708c62-bbdf-4462-8122-abf96a67c662 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:36.754217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c708c62-bbdf-4462-8122-abf96a67c662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:36.770702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f66a0-0b00-49cd-b6cb-268b7d41d271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:36.771638 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f66a0-0b00-49cd-b6cb-268b7d41d271 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:36.772128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f66a0-0b00-49cd-b6cb-268b7d41d271 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:36.772708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f66a0-0b00-49cd-b6cb-268b7d41d271 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:36.773597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f66a0-0b00-49cd-b6cb-268b7d41d271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:36.776299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f66a0-0b00-49cd-b6cb-268b7d41d271 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:36.795119 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6c708c62-bbdf-4462-8122-abf96a67c662 None None] MFA Rules not processed for user `698ccea06fb24beea18c679d53f907a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:36.802886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6c708c62-bbdf-4462-8122-abf96a67c662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:36.823684 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1126/2243] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:56:36 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:36.827388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f66a0-0b00-49cd-b6cb-268b7d41d271 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:36.848290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f66a0-0b00-49cd-b6cb-268b7d41d271 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:36.861837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f6472-b357-43af-b442-abf812746180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:36.862688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f6472-b357-43af-b442-abf812746180 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:36.863102 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f6472-b357-43af-b442-abf812746180 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:36.863485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f6472-b357-43af-b442-abf812746180 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:36.864342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f6472-b357-43af-b442-abf812746180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:36.869341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f6472-b357-43af-b442-abf812746180 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:36.886538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1127/2244] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:56:36.892209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f6472-b357-43af-b442-abf812746180 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:36.909942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f6472-b357-43af-b442-abf812746180 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:36.918602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1118/2245] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:56:36.918824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a080c2e-48f6-478f-8118-99982601d468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:36.918824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a080c2e-48f6-478f-8118-99982601d468 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:36.918824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a080c2e-48f6-478f-8118-99982601d468 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:36.918824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a080c2e-48f6-478f-8118-99982601d468 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:36.919205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a080c2e-48f6-478f-8118-99982601d468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:36.921993 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a080c2e-48f6-478f-8118-99982601d468 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:36.946922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1128/2246] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:36.949480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c708c62-bbdf-4462-8122-abf96a67c662 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:36.960618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c708c62-bbdf-4462-8122-abf96a67c662 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:36.965271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e725f21-1d12-41b0-87a4-fc37bdcfc2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:36.966149 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e725f21-1d12-41b0-87a4-fc37bdcfc2f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:36.966439 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e725f21-1d12-41b0-87a4-fc37bdcfc2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:36.966713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e725f21-1d12-41b0-87a4-fc37bdcfc2f6 admin admin] PATH_INFO: `/v3/projects/45a6637cc9e94be9871b167386a7de1a/users/8cf8292af1384c59b1139c8d8d8236db/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:36.989727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e725f21-1d12-41b0-87a4-fc37bdcfc2f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a6637cc9e94be9871b167386a7de1a, user_id=8cf8292af1384c59b1139c8d8d8236db, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:36.992677 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e725f21-1d12-41b0-87a4-fc37bdcfc2f6 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:37.021999 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1119/2247] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:36 2022] PUT /identity/v3/projects/45a6637cc9e94be9871b167386a7de1a/users/8cf8292af1384c59b1139c8d8d8236db/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:37.032400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c61a9b-a1d8-45ee-b3c4-36f6f4c94fa1 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:37.032699 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c61a9b-a1d8-45ee-b3c4-36f6f4c94fa1 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:37.032978 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c61a9b-a1d8-45ee-b3c4-36f6f4c94fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:37.099952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-b3c61a9b-a1d8-45ee-b3c4-36f6f4c94fa1 None None] MFA Rules not processed for user `8cf8292af1384c59b1139c8d8d8236db`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:37.101233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-b3c61a9b-a1d8-45ee-b3c4-36f6f4c94fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:37.243153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:37.252205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1129/2248] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:37.255259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e725f21-1d12-41b0-87a4-fc37bdcfc2f6 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:37.369849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e725f21-1d12-41b0-87a4-fc37bdcfc2f6 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:37.383257 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4ba73-5a55-4fe1-adbe-07171984a393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:37.386956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4ba73-5a55-4fe1-adbe-07171984a393 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:37.386956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4ba73-5a55-4fe1-adbe-07171984a393 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:37.386956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4ba73-5a55-4fe1-adbe-07171984a393 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:37.389009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4ba73-5a55-4fe1-adbe-07171984a393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:37.393435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4ba73-5a55-4fe1-adbe-07171984a393 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:37.442470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e52b1c-1fde-493b-a2c9-eba4ef39bec7 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:37.443656 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e52b1c-1fde-493b-a2c9-eba4ef39bec7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:37.444134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e52b1c-1fde-493b-a2c9-eba4ef39bec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:37.477942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1120/2249] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:56:37 2022] POST /identity/v3/projects => generated 371 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:37.486952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4ba73-5a55-4fe1-adbe-07171984a393 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:37.504216 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-82e52b1c-1fde-493b-a2c9-eba4ef39bec7 None None] MFA Rules not processed for user `a1274435c6774faa9eeeac6726b52b47`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:37.505463 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-82e52b1c-1fde-493b-a2c9-eba4ef39bec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:37.517859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:37.642330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1130/2250] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:37.642538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4ba73-5a55-4fe1-adbe-07171984a393 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:37.643871 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e52b1c-1fde-493b-a2c9-eba4ef39bec7 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:37.645251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-82e52b1c-1fde-493b-a2c9-eba4ef39bec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:37.649686 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b2084-9fd6-495b-9360-5df7e02c3b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:37.650696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b2084-9fd6-495b-9360-5df7e02c3b09 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:37.650990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b2084-9fd6-495b-9360-5df7e02c3b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:37.651274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b2084-9fd6-495b-9360-5df7e02c3b09 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:37.651974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b2084-9fd6-495b-9360-5df7e02c3b09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:37.654748 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b2084-9fd6-495b-9360-5df7e02c3b09 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:37.675921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e52b1c-1fde-493b-a2c9-eba4ef39bec7 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:37.691782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6563f59-29cb-4122-ac34-a6faf537a420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:37.694315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6563f59-29cb-4122-ac34-a6faf537a420 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:37.694706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6563f59-29cb-4122-ac34-a6faf537a420 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:37.694984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6563f59-29cb-4122-ac34-a6faf537a420 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:37.696039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6563f59-29cb-4122-ac34-a6faf537a420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:37.701153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6563f59-29cb-4122-ac34-a6faf537a420 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:37.704674 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1121/2251] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:56:37 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:37.711921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b2084-9fd6-495b-9360-5df7e02c3b09 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:37.734732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b2084-9fd6-495b-9360-5df7e02c3b09 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:37.765102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb73732-3575-4007-ba01-9a60037bbd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:37.765786 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb73732-3575-4007-ba01-9a60037bbd93 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:37.766069 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb73732-3575-4007-ba01-9a60037bbd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:37.770323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1131/2252] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:56:37 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:56:37.770518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb73732-3575-4007-ba01-9a60037bbd93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:37.774839 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb73732-3575-4007-ba01-9a60037bbd93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:37.776269 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6563f59-29cb-4122-ac34-a6faf537a420 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:37.777753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb73732-3575-4007-ba01-9a60037bbd93 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:37.806201 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1122/2253] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:37.810526 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb73732-3575-4007-ba01-9a60037bbd93 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:37.833159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb73732-3575-4007-ba01-9a60037bbd93 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:37.953378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6563f59-29cb-4122-ac34-a6faf537a420 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:37.967154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1559c53-f253-487e-8f83-964df60d999e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:37.968109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1559c53-f253-487e-8f83-964df60d999e admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:37.968551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1559c53-f253-487e-8f83-964df60d999e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:37.969167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1559c53-f253-487e-8f83-964df60d999e admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:37.971471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1559c53-f253-487e-8f83-964df60d999e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:37.973933 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1559c53-f253-487e-8f83-964df60d999e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:37.999501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036fa62-7517-4ca9-9353-398816fe279f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:38.000468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036fa62-7517-4ca9-9353-398816fe279f admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:38.001187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036fa62-7517-4ca9-9353-398816fe279f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:38.001887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036fa62-7517-4ca9-9353-398816fe279f admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:38.002776 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036fa62-7517-4ca9-9353-398816fe279f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:38.006696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1132/2254] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:56:37 2022] POST /identity/v3/users => generated 298 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:38.008734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036fa62-7517-4ca9-9353-398816fe279f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:38.009454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1559c53-f253-487e-8f83-964df60d999e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:38.026784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1559c53-f253-487e-8f83-964df60d999e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:38.034063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d0ef2-c343-4d58-a42b-92deb7d3a717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:38.034810 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d0ef2-c343-4d58-a42b-92deb7d3a717 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:38.035445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d0ef2-c343-4d58-a42b-92deb7d3a717 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:38.035624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d0ef2-c343-4d58-a42b-92deb7d3a717 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:38.036390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d0ef2-c343-4d58-a42b-92deb7d3a717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:38.037500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1123/2255] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:38.038413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036fa62-7517-4ca9-9353-398816fe279f admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:38.038520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d0ef2-c343-4d58-a42b-92deb7d3a717 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:38.055036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1133/2256] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:38.059326 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d0ef2-c343-4d58-a42b-92deb7d3a717 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:38.064045 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036fa62-7517-4ca9-9353-398816fe279f admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:38.082977 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac9994-a1b9-4f6b-ba97-a13b5fdc2e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:38.083804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac9994-a1b9-4f6b-ba97-a13b5fdc2e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:38.084085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac9994-a1b9-4f6b-ba97-a13b5fdc2e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:38.084361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac9994-a1b9-4f6b-ba97-a13b5fdc2e6c admin admin] PATH_INFO: `/v3/projects/903adcf7257046f88317019c012d32d3/users/ecdeb9b391a14e9bb58ce29010c6f22e/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:38.084935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d0ef2-c343-4d58-a42b-92deb7d3a717 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:38.100866 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9732e92-8037-46fb-b87f-1de3ba8bbfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:38.101923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9732e92-8037-46fb-b87f-1de3ba8bbfce admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:38.102248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9732e92-8037-46fb-b87f-1de3ba8bbfce admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:38.102599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9732e92-8037-46fb-b87f-1de3ba8bbfce admin admin] PATH_INFO: `/v3/projects/1f4275b742b84ecc97c6aff544e82d88/users/c04a766278dd451da58e003d989f9959/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:38.125148 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac9994-a1b9-4f6b-ba97-a13b5fdc2e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=903adcf7257046f88317019c012d32d3, user_id=ecdeb9b391a14e9bb58ce29010c6f22e, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:38.128458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac9994-a1b9-4f6b-ba97-a13b5fdc2e6c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:38.140383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9732e92-8037-46fb-b87f-1de3ba8bbfce admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4275b742b84ecc97c6aff544e82d88, user_id=c04a766278dd451da58e003d989f9959, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:38.145074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9732e92-8037-46fb-b87f-1de3ba8bbfce admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:38.158386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1124/2257] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:38 2022] PUT /identity/v3/projects/903adcf7257046f88317019c012d32d3/users/ecdeb9b391a14e9bb58ce29010c6f22e/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:38.174114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d1a26-ae32-419e-a96b-d230ca38eee8 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:38.175420 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d1a26-ae32-419e-a96b-d230ca38eee8 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:38.175760 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d1a26-ae32-419e-a96b-d230ca38eee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:38.191283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1134/2258] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:38 2022] PUT /identity/v3/projects/1f4275b742b84ecc97c6aff544e82d88/users/c04a766278dd451da58e003d989f9959/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:56:38.197508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9732e92-8037-46fb-b87f-1de3ba8bbfce admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:38.241469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-140d1a26-ae32-419e-a96b-d230ca38eee8 None None] MFA Rules not processed for user `ecdeb9b391a14e9bb58ce29010c6f22e`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:38.244003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-140d1a26-ae32-419e-a96b-d230ca38eee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:38.258212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:38.332597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9732e92-8037-46fb-b87f-1de3ba8bbfce admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:38.340428 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3fc5b-4f94-4c51-b863-a1c6a4ae52b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:38.341498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3fc5b-4f94-4c51-b863-a1c6a4ae52b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:38.341855 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3fc5b-4f94-4c51-b863-a1c6a4ae52b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:38.342207 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3fc5b-4f94-4c51-b863-a1c6a4ae52b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:38.343159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3fc5b-4f94-4c51-b863-a1c6a4ae52b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:38.353866 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3fc5b-4f94-4c51-b863-a1c6a4ae52b9 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:38.370254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1135/2259] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:38.373916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3fc5b-4f94-4c51-b863-a1c6a4ae52b9 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:38.382077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:38.392115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3fc5b-4f94-4c51-b863-a1c6a4ae52b9 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:38.406932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1125/2260] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:38.407936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccc88d-7f3e-4fc0-8bb2-0fe88f690285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:38.409097 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d1a26-ae32-419e-a96b-d230ca38eee8 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:38.410357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccc88d-7f3e-4fc0-8bb2-0fe88f690285 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:38.410936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccc88d-7f3e-4fc0-8bb2-0fe88f690285 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:38.411650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccc88d-7f3e-4fc0-8bb2-0fe88f690285 admin admin] PATH_INFO: `/v3/projects/1f4275b742b84ecc97c6aff544e82d88/users/c04a766278dd451da58e003d989f9959/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:38.416749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccc88d-7f3e-4fc0-8bb2-0fe88f690285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4275b742b84ecc97c6aff544e82d88, user_id=c04a766278dd451da58e003d989f9959, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:38.424128 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d1a26-ae32-419e-a96b-d230ca38eee8 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:38.424325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccc88d-7f3e-4fc0-8bb2-0fe88f690285 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:38.430569 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10930271-acf0-4e8f-940e-bb0318e1385e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:38.431376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10930271-acf0-4e8f-940e-bb0318e1385e admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:38.431870 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10930271-acf0-4e8f-940e-bb0318e1385e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:38.432435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10930271-acf0-4e8f-940e-bb0318e1385e admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:38.433544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10930271-acf0-4e8f-940e-bb0318e1385e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:38.435734 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10930271-acf0-4e8f-940e-bb0318e1385e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:38.451527 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1136/2261] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:38 2022] PUT /identity/v3/projects/1f4275b742b84ecc97c6aff544e82d88/users/c04a766278dd451da58e003d989f9959/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:38.482530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1126/2262] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:56:38 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:56:38.487830 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccc88d-7f3e-4fc0-8bb2-0fe88f690285 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:38.627275 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccc88d-7f3e-4fc0-8bb2-0fe88f690285 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:38.745603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc64c4-785b-4864-8c3e-e740a474ba35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:38.746494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc64c4-785b-4864-8c3e-e740a474ba35 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:38.746875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc64c4-785b-4864-8c3e-e740a474ba35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:38.747482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc64c4-785b-4864-8c3e-e740a474ba35 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:38.748809 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc64c4-785b-4864-8c3e-e740a474ba35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:38.751829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc64c4-785b-4864-8c3e-e740a474ba35 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:38.799776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1137/2263] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:56:38 2022] POST /identity/v3/users => generated 295 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:38.801340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10930271-acf0-4e8f-940e-bb0318e1385e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:38.816506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10930271-acf0-4e8f-940e-bb0318e1385e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:38.829924 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37209e9-76f8-43d1-a89f-dbe049ba665c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:38.830932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37209e9-76f8-43d1-a89f-dbe049ba665c admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:38.831308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37209e9-76f8-43d1-a89f-dbe049ba665c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:38.831740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37209e9-76f8-43d1-a89f-dbe049ba665c admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:38.834018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37209e9-76f8-43d1-a89f-dbe049ba665c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:38.836245 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37209e9-76f8-43d1-a89f-dbe049ba665c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:38.891764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1127/2264] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:38.895634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc64c4-785b-4864-8c3e-e740a474ba35 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:38.915803 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc64c4-785b-4864-8c3e-e740a474ba35 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:38.923844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0ea20-14cf-4a91-95d3-b09e5857f22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:38.925496 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0ea20-14cf-4a91-95d3-b09e5857f22a admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:38.925887 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0ea20-14cf-4a91-95d3-b09e5857f22a admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:38.926316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0ea20-14cf-4a91-95d3-b09e5857f22a admin admin] PATH_INFO: `/v3/projects/c9ac6b86eab54c57af50d611ee0ded21/users/e2a96ea0eac445c68db9394339942204/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:38.956170 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0ea20-14cf-4a91-95d3-b09e5857f22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ac6b86eab54c57af50d611ee0ded21, user_id=e2a96ea0eac445c68db9394339942204, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:38.959719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0ea20-14cf-4a91-95d3-b09e5857f22a admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:38.988175 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1138/2265] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:38 2022] PUT /identity/v3/projects/c9ac6b86eab54c57af50d611ee0ded21/users/e2a96ea0eac445c68db9394339942204/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:38.992529 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37209e9-76f8-43d1-a89f-dbe049ba665c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:39.006240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37209e9-76f8-43d1-a89f-dbe049ba665c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:39.134206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da696a-6bc8-4498-811f-a45edf92deff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:39.135055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da696a-6bc8-4498-811f-a45edf92deff admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:39.135222 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da696a-6bc8-4498-811f-a45edf92deff admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:39.135496 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da696a-6bc8-4498-811f-a45edf92deff admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:39.136938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da696a-6bc8-4498-811f-a45edf92deff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:39.138666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da696a-6bc8-4498-811f-a45edf92deff admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:39.168662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1128/2266] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:39.170015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0ea20-14cf-4a91-95d3-b09e5857f22a admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:39.186108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0ea20-14cf-4a91-95d3-b09e5857f22a admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:39.194330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f689053-4ae1-464d-8547-942160adbff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:39.195994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f689053-4ae1-464d-8547-942160adbff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:39.196493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f689053-4ae1-464d-8547-942160adbff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:39.196985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f689053-4ae1-464d-8547-942160adbff4 admin admin] PATH_INFO: `/v3/projects/c9ac6b86eab54c57af50d611ee0ded21/users/e2a96ea0eac445c68db9394339942204/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:39.199716 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f689053-4ae1-464d-8547-942160adbff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ac6b86eab54c57af50d611ee0ded21, user_id=e2a96ea0eac445c68db9394339942204, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:39.203582 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f689053-4ae1-464d-8547-942160adbff4 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:39.226275 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1139/2267] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:39 2022] PUT /identity/v3/projects/c9ac6b86eab54c57af50d611ee0ded21/users/e2a96ea0eac445c68db9394339942204/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:39.238029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64603c5e-fa89-4b3c-85de-104abdd3dca9 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:39.238352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64603c5e-fa89-4b3c-85de-104abdd3dca9 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:39.238594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64603c5e-fa89-4b3c-85de-104abdd3dca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:39.315851 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-64603c5e-fa89-4b3c-85de-104abdd3dca9 None None] MFA Rules not processed for user `e2a96ea0eac445c68db9394339942204`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:39.317345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-64603c5e-fa89-4b3c-85de-104abdd3dca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:39.507927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1129/2268] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:39.522156 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f689053-4ae1-464d-8547-942160adbff4 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:39.524008 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8f689053-4ae1-464d-8547-942160adbff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:39.531082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:39.611051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f689053-4ae1-464d-8547-942160adbff4 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:39.615332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ad0b6-2e02-4243-9522-9fc8db37d957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:39.615999 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ad0b6-2e02-4243-9522-9fc8db37d957 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:39.616282 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ad0b6-2e02-4243-9522-9fc8db37d957 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:39.616559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ad0b6-2e02-4243-9522-9fc8db37d957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:39.692467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ad0b6-2e02-4243-9522-9fc8db37d957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:39.694065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ad0b6-2e02-4243-9522-9fc8db37d957 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:39.704901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1140/2269] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:56:43.751776 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64603c5e-fa89-4b3c-85de-104abdd3dca9 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:43.782895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64603c5e-fa89-4b3c-85de-104abdd3dca9 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:43.789859 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cdece3-f235-4220-8679-fc52752ed1fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:43.790664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdece3-f235-4220-8679-fc52752ed1fc service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:43.791077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdece3-f235-4220-8679-fc52752ed1fc service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:43.791450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdece3-f235-4220-8679-fc52752ed1fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:43.856510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:43.946351 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cdece3-f235-4220-8679-fc52752ed1fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:43.948209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cdece3-f235-4220-8679-fc52752ed1fc service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:43.959380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1130/2270] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:46.657173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ad0b6-2e02-4243-9522-9fc8db37d957 service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:46.703341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ad0b6-2e02-4243-9522-9fc8db37d957 service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:46.760096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbb9a9-31e6-4c64-8bf8-9be552872841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:46.760788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbb9a9-31e6-4c64-8bf8-9be552872841 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:46.761086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbb9a9-31e6-4c64-8bf8-9be552872841 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:46.761376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbb9a9-31e6-4c64-8bf8-9be552872841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:46.794696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbb9a9-31e6-4c64-8bf8-9be552872841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:46.811220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbb9a9-31e6-4c64-8bf8-9be552872841 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:46.903383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1141/2271] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:48.721246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78365bff-e761-452a-885c-94474441a80c None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:48.721769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78365bff-e761-452a-885c-94474441a80c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:48.721853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78365bff-e761-452a-885c-94474441a80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:48.819535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-78365bff-e761-452a-885c-94474441a80c None None] MFA Rules not processed for user `c04a766278dd451da58e003d989f9959`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:48.822127 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-78365bff-e761-452a-885c-94474441a80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:49.015187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1131/2272] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:49.024512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbb9a9-31e6-4c64-8bf8-9be552872841 service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:49.026664 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-98fbb9a9-31e6-4c64-8bf8-9be552872841 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:49.123328 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbb9a9-31e6-4c64-8bf8-9be552872841 service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:49.131330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788efcdd-ccba-43cb-afee-6e9787099351 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '666ade4ce1fe447487998d086779389b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:49.132149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788efcdd-ccba-43cb-afee-6e9787099351 service glance] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:49.132524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788efcdd-ccba-43cb-afee-6e9787099351 service glance] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:49.132799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788efcdd-ccba-43cb-afee-6e9787099351 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:49.208623 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:49.221381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788efcdd-ccba-43cb-afee-6e9787099351 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:49.223341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788efcdd-ccba-43cb-afee-6e9787099351 service glance] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:49.246680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1142/2273] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:49.449590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e352e3-85c0-461f-bd59-903c9ded1f3f None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:49.450155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e352e3-85c0-461f-bd59-903c9ded1f3f None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:49.450417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e352e3-85c0-461f-bd59-903c9ded1f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:49.484517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-18e352e3-85c0-461f-bd59-903c9ded1f3f None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:49.485877 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-18e352e3-85c0-461f-bd59-903c9ded1f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:49.590806 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1132/2274] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 13:56:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:49.614176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788efcdd-ccba-43cb-afee-6e9787099351 service glance] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:49.616553 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-788efcdd-ccba-43cb-afee-6e9787099351 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:49.769128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788efcdd-ccba-43cb-afee-6e9787099351 service glance] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:49.778399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea6b3e-2cb0-479d-882a-5d347e795d88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:49.780004 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea6b3e-2cb0-479d-882a-5d347e795d88 service swift] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:49.780996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea6b3e-2cb0-479d-882a-5d347e795d88 service swift] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:49.781362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea6b3e-2cb0-479d-882a-5d347e795d88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:49.794164 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea6b3e-2cb0-479d-882a-5d347e795d88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:49.796028 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea6b3e-2cb0-479d-882a-5d347e795d88 service swift] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:49.820851 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1143/2275] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 13:56:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:56:51.541401 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e352e3-85c0-461f-bd59-903c9ded1f3f None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:51.695051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:51.715813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e352e3-85c0-461f-bd59-903c9ded1f3f None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:51.727817 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:51.728822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:51.729208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:51.729588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] PATH_INFO: `/v3/users/8cf8292af1384c59b1139c8d8d8236db` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:51.731854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cf8292af1384c59b1139c8d8d8236db)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:51.734067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:51.851143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cf8292af1384c59b1139c8d8d8236db'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:51.861545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cf8292af1384c59b1139c8d8d8236db'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:51.862604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cf8292af1384c59b1139c8d8d8236db'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:51.889263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cf8292af1384c59b1139c8d8d8236db'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:51.957765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cf8292af1384c59b1139c8d8d8236db'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:51.984241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1133/2276] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:51 2022] DELETE /identity/v3/users/8cf8292af1384c59b1139c8d8d8236db => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:51.990128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b94647-f34a-4317-929f-ddc1cb053619 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:51.990750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b94647-f34a-4317-929f-ddc1cb053619 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:51.990750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b94647-f34a-4317-929f-ddc1cb053619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:52.054410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-43b94647-f34a-4317-929f-ddc1cb053619 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:52.055822 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-43b94647-f34a-4317-929f-ddc1cb053619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:52.240868 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1144/2277] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:52.252586 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:52.254329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:52.394343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a615d-404a-4f87-8f56-865eb7ea6d8e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:52.404138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8cee6-db9e-43f7-8437-9309ca821cab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:52.404825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8cee6-db9e-43f7-8437-9309ca821cab service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:52.405108 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8cee6-db9e-43f7-8437-9309ca821cab service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:52.405385 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8cee6-db9e-43f7-8437-9309ca821cab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:52.427079 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8cee6-db9e-43f7-8437-9309ca821cab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:52.428549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8cee6-db9e-43f7-8437-9309ca821cab service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:52.475111 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1134/2278] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:53.401692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b94647-f34a-4317-929f-ddc1cb053619 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:53.403202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-43b94647-f34a-4317-929f-ddc1cb053619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:53.420333 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b94647-f34a-4317-929f-ddc1cb053619 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:53.441461 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:53.442345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:53.442629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:53.443107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] PATH_INFO: `/v3/projects/45a6637cc9e94be9871b167386a7de1a` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:53.447988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a6637cc9e94be9871b167386a7de1a)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:53.450142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:53.645398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a6637cc9e94be9871b167386a7de1a', 'request_id': 'req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:53.661535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a6637cc9e94be9871b167386a7de1a', 'request_id': 'req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:53.696859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a6637cc9e94be9871b167386a7de1a', 'request_id': 'req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:53.743966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] The token cache is being invalidate because project 45a6637cc9e94be9871b167386a7de1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:53.743966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:53.744867 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:53.747605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1145/2279] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:53 2022] DELETE /identity/v3/projects/45a6637cc9e94be9871b167386a7de1a => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:53.750679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8cee6-db9e-43f7-8437-9309ca821cab service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:53.770056 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8cee6-db9e-43f7-8437-9309ca821cab service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:53.889156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2222e-3943-4aa7-a50f-eaba61ada0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:53.890913 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2222e-3943-4aa7-a50f-eaba61ada0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:53.890913 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2222e-3943-4aa7-a50f-eaba61ada0da admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:53.890913 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2222e-3943-4aa7-a50f-eaba61ada0da admin admin] PATH_INFO: `/v3/users/ecdeb9b391a14e9bb58ce29010c6f22e` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:53.892608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2222e-3943-4aa7-a50f-eaba61ada0da admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecdeb9b391a14e9bb58ce29010c6f22e)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:53.896607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2222e-3943-4aa7-a50f-eaba61ada0da admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:53.928786 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:54.046959 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-eba2222e-3943-4aa7-a50f-eaba61ada0da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecdeb9b391a14e9bb58ce29010c6f22e'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:54.064186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-eba2222e-3943-4aa7-a50f-eaba61ada0da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecdeb9b391a14e9bb58ce29010c6f22e'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:54.064877 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-eba2222e-3943-4aa7-a50f-eaba61ada0da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecdeb9b391a14e9bb58ce29010c6f22e'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:54.089954 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-eba2222e-3943-4aa7-a50f-eaba61ada0da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecdeb9b391a14e9bb58ce29010c6f22e'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:54.155041 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-eba2222e-3943-4aa7-a50f-eaba61ada0da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecdeb9b391a14e9bb58ce29010c6f22e'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:54.167990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1135/2280] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:53 2022] DELETE /identity/v3/users/ecdeb9b391a14e9bb58ce29010c6f22e => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:56:54.255659 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:54.257234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:54.292811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:54.303322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac281d6b-cd7b-4b76-bf7a-e3f0115d5583 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:54.484326 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:54.486435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:54.490176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:54.490531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] PATH_INFO: `/v3/users/b7f3493e90ae4776986b862237846099` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:54.496744 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7f3493e90ae4776986b862237846099)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:54.506368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:54.660599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7f3493e90ae4776986b862237846099'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:54.749534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7f3493e90ae4776986b862237846099'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:54.767621 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7f3493e90ae4776986b862237846099'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:54.768396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7f3493e90ae4776986b862237846099'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:54.769033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7f3493e90ae4776986b862237846099'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:54.825404 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1146/2281] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:54 2022] DELETE /identity/v3/users/b7f3493e90ae4776986b862237846099 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:54.827188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f686a-dd58-44ed-89fa-2584b8e1b92c None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:54.827689 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f686a-dd58-44ed-89fa-2584b8e1b92c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:54.827778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f686a-dd58-44ed-89fa-2584b8e1b92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:54.934973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-fe0f686a-dd58-44ed-89fa-2584b8e1b92c None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:54.977851 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-fe0f686a-dd58-44ed-89fa-2584b8e1b92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:55.132204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:55.134595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:55.181902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1136/2282] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:55.188912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f686a-dd58-44ed-89fa-2584b8e1b92c None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:55.191881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-fe0f686a-dd58-44ed-89fa-2584b8e1b92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:55.309539 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea03d0-765e-43da-bdc5-3fe7e4bba5a6 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:55.320548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:55.324542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:55.325880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:55.326699 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] PATH_INFO: `/v3/projects/903adcf7257046f88317019c012d32d3` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:55.329938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=903adcf7257046f88317019c012d32d3)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:55.334379 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:55.455714 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:55.488630 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f686a-dd58-44ed-89fa-2584b8e1b92c None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:55.491266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '903adcf7257046f88317019c012d32d3', 'request_id': 'req-95e5ec94-6480-4adf-be8c-03d038abdc31'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:55.510037 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '903adcf7257046f88317019c012d32d3', 'request_id': 'req-95e5ec94-6480-4adf-be8c-03d038abdc31'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:55.537006 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '903adcf7257046f88317019c012d32d3', 'request_id': 'req-95e5ec94-6480-4adf-be8c-03d038abdc31'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:55.554678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] The token cache is being invalidate because project 903adcf7257046f88317019c012d32d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:55.555032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:55.555817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:55.559303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1147/2283] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:55 2022] DELETE /identity/v3/projects/903adcf7257046f88317019c012d32d3 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:55.566969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:55.571852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:55.752825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26862597-025c-40da-a992-aa02ee3ef3a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:55.756140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26862597-025c-40da-a992-aa02ee3ef3a7 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:55.760495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26862597-025c-40da-a992-aa02ee3ef3a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:55.761003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26862597-025c-40da-a992-aa02ee3ef3a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:55.768136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5ec94-6480-4adf-be8c-03d038abdc31 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:55.785706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:55.788108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:55.788315 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-26862597-025c-40da-a992-aa02ee3ef3a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:55.788692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:55.789454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] PATH_INFO: `/v3/users/e2a96ea0eac445c68db9394339942204` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:55.795213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2a96ea0eac445c68db9394339942204)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:55.800561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:55.827781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:55.841078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26862597-025c-40da-a992-aa02ee3ef3a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:55.842974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26862597-025c-40da-a992-aa02ee3ef3a7 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:55.868897 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1137/2284] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:56.021942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2a96ea0eac445c68db9394339942204'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:56.074918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2a96ea0eac445c68db9394339942204'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:56.074918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2a96ea0eac445c68db9394339942204'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:56.074918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2a96ea0eac445c68db9394339942204'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:56.074918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2a96ea0eac445c68db9394339942204'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:56.120626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1148/2285] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:56:55 2022] DELETE /identity/v3/users/e2a96ea0eac445c68db9394339942204 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:56.687948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26862597-025c-40da-a992-aa02ee3ef3a7 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:56.689577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-26862597-025c-40da-a992-aa02ee3ef3a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:56.716301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26862597-025c-40da-a992-aa02ee3ef3a7 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:56.956550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dbb33-e798-4677-a9cb-47b20d845508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:56.961936 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dbb33-e798-4677-a9cb-47b20d845508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:56.963075 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dbb33-e798-4677-a9cb-47b20d845508 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:56.964634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dbb33-e798-4677-a9cb-47b20d845508 admin admin] PATH_INFO: `/v3/projects/957af36a92db42a19f014ab2948e5516` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:56.977003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dbb33-e798-4677-a9cb-47b20d845508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=957af36a92db42a19f014ab2948e5516)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:56.991887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dbb33-e798-4677-a9cb-47b20d845508 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:57.021851 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:57.027205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:57.085385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c31cfa-7028-4041-8392-eec9285ff4aa admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:57.112339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855c0c5-e5b6-4738-8574-674b06bb0c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:57.113439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855c0c5-e5b6-4738-8574-674b06bb0c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:57.113826 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855c0c5-e5b6-4738-8574-674b06bb0c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:57.114695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855c0c5-e5b6-4738-8574-674b06bb0c21 admin admin] PATH_INFO: `/v3/projects/c9ac6b86eab54c57af50d611ee0ded21` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:57.118267 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855c0c5-e5b6-4738-8574-674b06bb0c21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9ac6b86eab54c57af50d611ee0ded21)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:57.125424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855c0c5-e5b6-4738-8574-674b06bb0c21 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:57.167281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-429dbb33-e798-4677-a9cb-47b20d845508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '957af36a92db42a19f014ab2948e5516', 'request_id': 'req-429dbb33-e798-4677-a9cb-47b20d845508'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:57.191845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-429dbb33-e798-4677-a9cb-47b20d845508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '957af36a92db42a19f014ab2948e5516', 'request_id': 'req-429dbb33-e798-4677-a9cb-47b20d845508'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:57.210657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:57.213758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-429dbb33-e798-4677-a9cb-47b20d845508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '957af36a92db42a19f014ab2948e5516', 'request_id': 'req-429dbb33-e798-4677-a9cb-47b20d845508'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:57.228209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-429dbb33-e798-4677-a9cb-47b20d845508 admin admin] The token cache is being invalidate because project 957af36a92db42a19f014ab2948e5516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:57.228894 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-429dbb33-e798-4677-a9cb-47b20d845508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:57.229726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-429dbb33-e798-4677-a9cb-47b20d845508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:57.231647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1138/2286] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:56 2022] DELETE /identity/v3/projects/957af36a92db42a19f014ab2948e5516 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:57.242079 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf0569-8c0b-41c6-b69e-0a4c3a6fcdab None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:57.243283 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf0569-8c0b-41c6-b69e-0a4c3a6fcdab None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:57.243731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf0569-8c0b-41c6-b69e-0a4c3a6fcdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:57.272032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4855c0c5-e5b6-4738-8574-674b06bb0c21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9ac6b86eab54c57af50d611ee0ded21', 'request_id': 'req-4855c0c5-e5b6-4738-8574-674b06bb0c21'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:57.298302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4855c0c5-e5b6-4738-8574-674b06bb0c21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9ac6b86eab54c57af50d611ee0ded21', 'request_id': 'req-4855c0c5-e5b6-4738-8574-674b06bb0c21'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:57.299641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-d9bf0569-8c0b-41c6-b69e-0a4c3a6fcdab None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:57.303140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d9bf0569-8c0b-41c6-b69e-0a4c3a6fcdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:57.321407 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4855c0c5-e5b6-4738-8574-674b06bb0c21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9ac6b86eab54c57af50d611ee0ded21', 'request_id': 'req-4855c0c5-e5b6-4738-8574-674b06bb0c21'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:57.354815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4855c0c5-e5b6-4738-8574-674b06bb0c21 admin admin] The token cache is being invalidate because project c9ac6b86eab54c57af50d611ee0ded21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:56:57.355276 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4855c0c5-e5b6-4738-8574-674b06bb0c21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:57.356043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4855c0c5-e5b6-4738-8574-674b06bb0c21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:56:57.357995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1149/2287] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:56:57 2022] DELETE /identity/v3/projects/c9ac6b86eab54c57af50d611ee0ded21 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:56:57.363245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc14ad-7f36-4946-9dcc-9b817100f59f None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:57.363783 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc14ad-7f36-4946-9dcc-9b817100f59f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:57.364172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc14ad-7f36-4946-9dcc-9b817100f59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:57.416689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f3cc14ad-7f36-4946-9dcc-9b817100f59f None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:57.418098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f3cc14ad-7f36-4946-9dcc-9b817100f59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:57.492372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1139/2288] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:57.493795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b43af-db4b-4f3a-bbc4-040bfa8d9016 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:57.494088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b43af-db4b-4f3a-bbc4-040bfa8d9016 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:57.494386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b43af-db4b-4f3a-bbc4-040bfa8d9016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:57.539327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a03b43af-db4b-4f3a-bbc4-040bfa8d9016 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:57.540757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a03b43af-db4b-4f3a-bbc4-040bfa8d9016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:57.558718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1140/2289] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:57.560460 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1150/2290] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:57.570603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc14ad-7f36-4946-9dcc-9b817100f59f None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:57.571123 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fd2c8-2619-4fd5-af4e-e02d71bed4d8 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:57.571415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fd2c8-2619-4fd5-af4e-e02d71bed4d8 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:57.571696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fd2c8-2619-4fd5-af4e-e02d71bed4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:57.599520 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3f6fd2c8-2619-4fd5-af4e-e02d71bed4d8 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:57.600925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3f6fd2c8-2619-4fd5-af4e-e02d71bed4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:57.602524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc14ad-7f36-4946-9dcc-9b817100f59f None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:57.617969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc0548-9168-4d46-8a0b-8c3fe3d48173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:57.619126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc0548-9168-4d46-8a0b-8c3fe3d48173 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:57.619648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc0548-9168-4d46-8a0b-8c3fe3d48173 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:57.620136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc0548-9168-4d46-8a0b-8c3fe3d48173 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:57.621096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc0548-9168-4d46-8a0b-8c3fe3d48173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:56:57.621651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc0548-9168-4d46-8a0b-8c3fe3d48173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:57.624307 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc0548-9168-4d46-8a0b-8c3fe3d48173 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:57.630930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1141/2291] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:57.632549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fd2c8-2619-4fd5-af4e-e02d71bed4d8 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:57.647986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fd2c8-2619-4fd5-af4e-e02d71bed4d8 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:57.653585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1151/2292] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:56:57.658199 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc0548-9168-4d46-8a0b-8c3fe3d48173 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:57.661443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e50b-2b39-4208-80b6-a2a5691522db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:57.662277 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e50b-2b39-4208-80b6-a2a5691522db admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:57.662641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e50b-2b39-4208-80b6-a2a5691522db admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:57.663025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e50b-2b39-4208-80b6-a2a5691522db admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:57.663982 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e50b-2b39-4208-80b6-a2a5691522db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:56:57.664361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e50b-2b39-4208-80b6-a2a5691522db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:57.666110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e50b-2b39-4208-80b6-a2a5691522db admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:57.685323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc0548-9168-4d46-8a0b-8c3fe3d48173 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:57.693029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:57.698918 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1bd00-befb-47e0-bd35-bf8aa14958af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:57.699730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1bd00-befb-47e0-bd35-bf8aa14958af admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:57.700123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1bd00-befb-47e0-bd35-bf8aa14958af admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:57.700704 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1bd00-befb-47e0-bd35-bf8aa14958af admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:57.701509 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1bd00-befb-47e0-bd35-bf8aa14958af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:57.704548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1142/2293] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:56:57.705886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1bd00-befb-47e0-bd35-bf8aa14958af admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:57.709426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e50b-2b39-4208-80b6-a2a5691522db admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:57.752014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e50b-2b39-4208-80b6-a2a5691522db admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:57.775978 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd89ce-674f-401f-9fc0-7f099611c410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:57.776382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd89ce-674f-401f-9fc0-7f099611c410 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:57.776674 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd89ce-674f-401f-9fc0-7f099611c410 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:57.776952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd89ce-674f-401f-9fc0-7f099611c410 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:57.777749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd89ce-674f-401f-9fc0-7f099611c410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:57.784589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd89ce-674f-401f-9fc0-7f099611c410 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:57.827888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1152/2294] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:56:57 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:57.835279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1bd00-befb-47e0-bd35-bf8aa14958af admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:57.845226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1143/2295] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:56:57 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:57.846490 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd89ce-674f-401f-9fc0-7f099611c410 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:57.966573 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd89ce-674f-401f-9fc0-7f099611c410 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:57.972252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0541ba0-4bb6-4675-8ff2-8028a5e53c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:57.972942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0541ba0-4bb6-4675-8ff2-8028a5e53c5c admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:57.973223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0541ba0-4bb6-4675-8ff2-8028a5e53c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:57.973498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0541ba0-4bb6-4675-8ff2-8028a5e53c5c admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:57.974192 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0541ba0-4bb6-4675-8ff2-8028a5e53c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:57.976214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0541ba0-4bb6-4675-8ff2-8028a5e53c5c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:58.013234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1144/2296] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:56:57 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:58.019255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0541ba0-4bb6-4675-8ff2-8028a5e53c5c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:58.044517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1bd00-befb-47e0-bd35-bf8aa14958af admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:58.044987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0541ba0-4bb6-4675-8ff2-8028a5e53c5c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:58.053930 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa84697-0d02-4c6c-8819-b92d496ccfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:58.054938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa84697-0d02-4c6c-8819-b92d496ccfa8 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:58.055668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e42c92-8c9a-4b2d-a975-1298ee3b6a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:58.056356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e42c92-8c9a-4b2d-a975-1298ee3b6a35 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:58.056727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e42c92-8c9a-4b2d-a975-1298ee3b6a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:58.056902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e42c92-8c9a-4b2d-a975-1298ee3b6a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:58.058924 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e42c92-8c9a-4b2d-a975-1298ee3b6a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:58.060092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e42c92-8c9a-4b2d-a975-1298ee3b6a35 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:58.065316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa84697-0d02-4c6c-8819-b92d496ccfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:58.065771 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa84697-0d02-4c6c-8819-b92d496ccfa8 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:58.068016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa84697-0d02-4c6c-8819-b92d496ccfa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:58.073548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa84697-0d02-4c6c-8819-b92d496ccfa8 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:58.104890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1145/2297] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:56:58.112218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e42c92-8c9a-4b2d-a975-1298ee3b6a35 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:58.137853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1153/2298] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:56:57 2022] POST /identity/v3/users => generated 287 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:56:58.139278 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa84697-0d02-4c6c-8819-b92d496ccfa8 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:58.154275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e42c92-8c9a-4b2d-a975-1298ee3b6a35 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:58.164597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa84697-0d02-4c6c-8819-b92d496ccfa8 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:58.169469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62108f36-88d3-4136-9634-7ed4c136fbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:58.170248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b21908-2ce0-44d8-9093-9ad6f8fd0858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:58.170726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62108f36-88d3-4136-9634-7ed4c136fbaa admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:58.171176 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62108f36-88d3-4136-9634-7ed4c136fbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:58.171318 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b21908-2ce0-44d8-9093-9ad6f8fd0858 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:58.171655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b21908-2ce0-44d8-9093-9ad6f8fd0858 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:58.172534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b21908-2ce0-44d8-9093-9ad6f8fd0858 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:58.173784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62108f36-88d3-4136-9634-7ed4c136fbaa admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:58.173969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b21908-2ce0-44d8-9093-9ad6f8fd0858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:58.174882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62108f36-88d3-4136-9634-7ed4c136fbaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:58.175784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b21908-2ce0-44d8-9093-9ad6f8fd0858 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:58.179458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62108f36-88d3-4136-9634-7ed4c136fbaa admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:58.183846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:58.200118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1154/2299] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:58.204564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b21908-2ce0-44d8-9093-9ad6f8fd0858 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:58.218274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1146/2300] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:58.222526 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62108f36-88d3-4136-9634-7ed4c136fbaa admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:58.237739 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b21908-2ce0-44d8-9093-9ad6f8fd0858 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:58.239546 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62108f36-88d3-4136-9634-7ed4c136fbaa admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:58.243261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c97ba-d9d0-4fdf-a694-64edce00e286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:58.243706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c97ba-d9d0-4fdf-a694-64edce00e286 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:58.244040 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c97ba-d9d0-4fdf-a694-64edce00e286 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:58.244328 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c97ba-d9d0-4fdf-a694-64edce00e286 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:58.245093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c97ba-d9d0-4fdf-a694-64edce00e286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:58.248333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56b82d-fe58-4d35-a005-8ebd0ef2064c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:58.248882 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c97ba-d9d0-4fdf-a694-64edce00e286 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:58.249345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56b82d-fe58-4d35-a005-8ebd0ef2064c admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:58.249624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56b82d-fe58-4d35-a005-8ebd0ef2064c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:58.249918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56b82d-fe58-4d35-a005-8ebd0ef2064c admin admin] PATH_INFO: `/v3/projects/7b7bcaa2e6e64f5990d3dfd701a0ec1a/users/3dbff413395d4159b961fca3f05e07d7/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:58.270903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1155/2301] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:56:58.274884 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c97ba-d9d0-4fdf-a694-64edce00e286 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:58.284555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56b82d-fe58-4d35-a005-8ebd0ef2064c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b7bcaa2e6e64f5990d3dfd701a0ec1a, user_id=3dbff413395d4159b961fca3f05e07d7, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:58.289370 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56b82d-fe58-4d35-a005-8ebd0ef2064c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:58.322003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1147/2302] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:58 2022] PUT /identity/v3/projects/7b7bcaa2e6e64f5990d3dfd701a0ec1a/users/3dbff413395d4159b961fca3f05e07d7/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:58.322764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c97ba-d9d0-4fdf-a694-64edce00e286 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:58.324330 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448a380-4f02-4531-a23f-4f2c583a14a1 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:58.324837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448a380-4f02-4531-a23f-4f2c583a14a1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:58.325123 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448a380-4f02-4531-a23f-4f2c583a14a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:58.360142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-d448a380-4f02-4531-a23f-4f2c583a14a1 None None] MFA Rules not processed for user `3dbff413395d4159b961fca3f05e07d7`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:58.361659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d448a380-4f02-4531-a23f-4f2c583a14a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:58.476971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21379542-8686-4531-aaed-7fb78ea1fc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:58.479075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21379542-8686-4531-aaed-7fb78ea1fc42 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:58.481835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21379542-8686-4531-aaed-7fb78ea1fc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:58.486301 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21379542-8686-4531-aaed-7fb78ea1fc42 admin admin] PATH_INFO: `/v3/projects/0197b4cff99649ef9da1e77b1461dd76/users/d5fabcc754684ded96c800be75777553/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:58.497151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1148/2303] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:58.506648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448a380-4f02-4531-a23f-4f2c583a14a1 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:58.511914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d448a380-4f02-4531-a23f-4f2c583a14a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:58.526602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21379542-8686-4531-aaed-7fb78ea1fc42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0197b4cff99649ef9da1e77b1461dd76, user_id=d5fabcc754684ded96c800be75777553, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:58.534473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21379542-8686-4531-aaed-7fb78ea1fc42 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:58.538527 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:58.574331 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1156/2304] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:56:58 2022] PUT /identity/v3/projects/0197b4cff99649ef9da1e77b1461dd76/users/d5fabcc754684ded96c800be75777553/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:56:58.576225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90af48e-1a17-4bcd-9bb9-7e93e0b1be01 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:58.576690 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90af48e-1a17-4bcd-9bb9-7e93e0b1be01 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:58.576982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90af48e-1a17-4bcd-9bb9-7e93e0b1be01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:58.606239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f90af48e-1a17-4bcd-9bb9-7e93e0b1be01 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:58.608041 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f90af48e-1a17-4bcd-9bb9-7e93e0b1be01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:58.664730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448a380-4f02-4531-a23f-4f2c583a14a1 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:58.708335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1157/2305] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:58.734143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90af48e-1a17-4bcd-9bb9-7e93e0b1be01 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:58.741673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f90af48e-1a17-4bcd-9bb9-7e93e0b1be01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:58.764286 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb7161-61c1-406a-abdb-6ec56bdc8223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:58.765250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb7161-61c1-406a-abdb-6ec56bdc8223 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:58.766092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb7161-61c1-406a-abdb-6ec56bdc8223 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:58.766654 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb7161-61c1-406a-abdb-6ec56bdc8223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:58.859416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90af48e-1a17-4bcd-9bb9-7e93e0b1be01 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:58.868110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4aea0e-4c30-4eee-ae66-80f8b526f935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:58.868768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4aea0e-4c30-4eee-ae66-80f8b526f935 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:58.869052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4aea0e-4c30-4eee-ae66-80f8b526f935 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:58.869327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4aea0e-4c30-4eee-ae66-80f8b526f935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:58.879034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb7161-61c1-406a-abdb-6ec56bdc8223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:58.880736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb7161-61c1-406a-abdb-6ec56bdc8223 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:58.890262 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4aea0e-4c30-4eee-ae66-80f8b526f935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:58.892082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4aea0e-4c30-4eee-ae66-80f8b526f935 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:58.892603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1149/2306] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:56:58.910543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1158/2307] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:56:59.504628 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba3a18-0320-48c9-be3f-b37d98303a9d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:59.505337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba3a18-0320-48c9-be3f-b37d98303a9d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:59.505337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba3a18-0320-48c9-be3f-b37d98303a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:59.526148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54afc9-eb06-4622-bd94-c973f6fd8b13 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:59.526697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54afc9-eb06-4622-bd94-c973f6fd8b13 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:59.526697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54afc9-eb06-4622-bd94-c973f6fd8b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:59.554500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-4a54afc9-eb06-4622-bd94-c973f6fd8b13 None None] MFA Rules not processed for user `0cc3e0105f2a472092423852b1f39979`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:59.555758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4a54afc9-eb06-4622-bd94-c973f6fd8b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:59.580990 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:59.604533 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-deba3a18-0320-48c9-be3f-b37d98303a9d None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:56:59.606429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-deba3a18-0320-48c9-be3f-b37d98303a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:59.684160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:56:59.707495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1159/2308] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:56:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:56:59.729986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1150/2309] 10.222.0.52 () {58 vars in 967 bytes} [Thu May 19 13:56:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:56:59.799596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54afc9-eb06-4622-bd94-c973f6fd8b13 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:56:59.802528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4a54afc9-eb06-4622-bd94-c973f6fd8b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:56:59.948558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54afc9-eb06-4622-bd94-c973f6fd8b13 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:56:59.954608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6ff15-4ce1-4de8-beaa-afdd2e22e300 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:56:59.955344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6ff15-4ce1-4de8-beaa-afdd2e22e300 service swift] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:56:59.955616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6ff15-4ce1-4de8-beaa-afdd2e22e300 service swift] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:56:59.955888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6ff15-4ce1-4de8-beaa-afdd2e22e300 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:56:59.968582 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6ff15-4ce1-4de8-beaa-afdd2e22e300 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:56:59.971213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6ff15-4ce1-4de8-beaa-afdd2e22e300 service swift] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:56:59.984272 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1160/2310] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 13:56:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:57:01.839580 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba3a18-0320-48c9-be3f-b37d98303a9d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:01.870893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba3a18-0320-48c9-be3f-b37d98303a9d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:01.887390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:01.892670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:01.894562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:01.895006 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] PATH_INFO: `/v3/users/3dbff413395d4159b961fca3f05e07d7` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:01.900724 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dbff413395d4159b961fca3f05e07d7)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:01.904860 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:02.008392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dbff413395d4159b961fca3f05e07d7'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:02.033873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dbff413395d4159b961fca3f05e07d7'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:02.035166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dbff413395d4159b961fca3f05e07d7'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:02.050056 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dbff413395d4159b961fca3f05e07d7'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:02.074498 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dbff413395d4159b961fca3f05e07d7'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:02.091152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1151/2311] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:57:01 2022] DELETE /identity/v3/users/3dbff413395d4159b961fca3f05e07d7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:57:02.097141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c9367-86aa-4d0e-bef9-68740f08e9a6 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:02.097661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c9367-86aa-4d0e-bef9-68740f08e9a6 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:02.097745 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c9367-86aa-4d0e-bef9-68740f08e9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:02.133941 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-338c9367-86aa-4d0e-bef9-68740f08e9a6 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:02.135231 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-338c9367-86aa-4d0e-bef9-68740f08e9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:02.286388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1161/2312] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:02.336899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:02.338504 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:02.525483 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636267c-ad9c-41be-a171-ac7fff21e08d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:02.536749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc180c4-c6e6-4dc6-ba6d-19949657c9bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:02.537658 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc180c4-c6e6-4dc6-ba6d-19949657c9bb service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:02.538108 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc180c4-c6e6-4dc6-ba6d-19949657c9bb service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:02.538484 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc180c4-c6e6-4dc6-ba6d-19949657c9bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:02.565908 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc180c4-c6e6-4dc6-ba6d-19949657c9bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:02.570572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc180c4-c6e6-4dc6-ba6d-19949657c9bb service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:02.588634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1152/2313] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:57:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:57:04.077552 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c9367-86aa-4d0e-bef9-68740f08e9a6 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:04.079213 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-338c9367-86aa-4d0e-bef9-68740f08e9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:04.097411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c9367-86aa-4d0e-bef9-68740f08e9a6 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:04.122251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83028d-7c77-4147-a761-4eac7d8ba83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:04.123165 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83028d-7c77-4147-a761-4eac7d8ba83b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:04.123454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83028d-7c77-4147-a761-4eac7d8ba83b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:04.123881 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83028d-7c77-4147-a761-4eac7d8ba83b admin admin] PATH_INFO: `/v3/projects/7b7bcaa2e6e64f5990d3dfd701a0ec1a` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:04.128847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83028d-7c77-4147-a761-4eac7d8ba83b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b7bcaa2e6e64f5990d3dfd701a0ec1a)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:04.135642 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83028d-7c77-4147-a761-4eac7d8ba83b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:04.256139 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:04.304833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-8c83028d-7c77-4147-a761-4eac7d8ba83b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b7bcaa2e6e64f5990d3dfd701a0ec1a', 'request_id': 'req-8c83028d-7c77-4147-a761-4eac7d8ba83b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:04.325288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-8c83028d-7c77-4147-a761-4eac7d8ba83b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b7bcaa2e6e64f5990d3dfd701a0ec1a', 'request_id': 'req-8c83028d-7c77-4147-a761-4eac7d8ba83b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:04.368279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-8c83028d-7c77-4147-a761-4eac7d8ba83b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b7bcaa2e6e64f5990d3dfd701a0ec1a', 'request_id': 'req-8c83028d-7c77-4147-a761-4eac7d8ba83b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:04.400066 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-8c83028d-7c77-4147-a761-4eac7d8ba83b admin admin] The token cache is being invalidate because project 7b7bcaa2e6e64f5990d3dfd701a0ec1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:57:04.400066 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-8c83028d-7c77-4147-a761-4eac7d8ba83b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:04.407540 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-8c83028d-7c77-4147-a761-4eac7d8ba83b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:04.412212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1162/2314] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:57:04 2022] DELETE /identity/v3/projects/7b7bcaa2e6e64f5990d3dfd701a0ec1a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:57:04.417918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650da3cf-f54b-4a1e-ae5c-1b8331d22c37 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:04.418385 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650da3cf-f54b-4a1e-ae5c-1b8331d22c37 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:04.418467 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650da3cf-f54b-4a1e-ae5c-1b8331d22c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:04.463081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-650da3cf-f54b-4a1e-ae5c-1b8331d22c37 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:04.472092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-650da3cf-f54b-4a1e-ae5c-1b8331d22c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:04.675895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:04.709354 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045cbc8-c551-4296-ab97-9f480d1eb9eb None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:04.709964 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045cbc8-c551-4296-ab97-9f480d1eb9eb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:04.710080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1153/2315] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:04.710482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045cbc8-c551-4296-ab97-9f480d1eb9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:04.756842 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-2045cbc8-c551-4296-ab97-9f480d1eb9eb None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:04.758299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2045cbc8-c551-4296-ab97-9f480d1eb9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:04.791248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1163/2316] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:04.793594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650da3cf-f54b-4a1e-ae5c-1b8331d22c37 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:04.811593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650da3cf-f54b-4a1e-ae5c-1b8331d22c37 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:04.861507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412a8ec-c938-4546-a3d6-577151b6337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:04.862204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412a8ec-c938-4546-a3d6-577151b6337a admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:04.862492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412a8ec-c938-4546-a3d6-577151b6337a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:04.867340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412a8ec-c938-4546-a3d6-577151b6337a admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:04.871506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412a8ec-c938-4546-a3d6-577151b6337a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:57:04.871648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412a8ec-c938-4546-a3d6-577151b6337a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:04.873343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412a8ec-c938-4546-a3d6-577151b6337a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:04.947020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045cbc8-c551-4296-ab97-9f480d1eb9eb None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:04.947229 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1154/2317] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:57:04.959130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045cbc8-c551-4296-ab97-9f480d1eb9eb None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:04.975508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd8b75-1584-40fa-9a24-011b2eac7db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:04.980340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd8b75-1584-40fa-9a24-011b2eac7db5 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:04.980773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd8b75-1584-40fa-9a24-011b2eac7db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:04.981206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd8b75-1584-40fa-9a24-011b2eac7db5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:04.983288 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd8b75-1584-40fa-9a24-011b2eac7db5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:04.990101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd8b75-1584-40fa-9a24-011b2eac7db5 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:05.083596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1164/2318] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:57:04 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:57:05.085335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412a8ec-c938-4546-a3d6-577151b6337a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:05.102933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412a8ec-c938-4546-a3d6-577151b6337a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:05.235221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd8b75-1584-40fa-9a24-011b2eac7db5 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:05.237339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-85bd8b75-1584-40fa-9a24-011b2eac7db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:05.311825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa41c8-ff50-4da3-9f81-5141cefefcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:05.313865 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa41c8-ff50-4da3-9f81-5141cefefcab admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:05.314476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa41c8-ff50-4da3-9f81-5141cefefcab admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:05.314960 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa41c8-ff50-4da3-9f81-5141cefefcab admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:05.316091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa41c8-ff50-4da3-9f81-5141cefefcab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:05.319500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa41c8-ff50-4da3-9f81-5141cefefcab admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:05.363820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1155/2319] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:57:05 2022] POST /identity/v3/users => generated 286 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:57:05.364936 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa41c8-ff50-4da3-9f81-5141cefefcab admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:05.404378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa41c8-ff50-4da3-9f81-5141cefefcab admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:05.417851 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf590f3-efb2-42f3-be34-791a381ae33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:05.418559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf590f3-efb2-42f3-be34-791a381ae33e admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:05.419164 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf590f3-efb2-42f3-be34-791a381ae33e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:05.419164 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf590f3-efb2-42f3-be34-791a381ae33e admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:05.419964 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf590f3-efb2-42f3-be34-791a381ae33e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:05.421597 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf590f3-efb2-42f3-be34-791a381ae33e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:05.444997 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1156/2320] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:57:05.447963 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf590f3-efb2-42f3-be34-791a381ae33e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:05.456807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd8b75-1584-40fa-9a24-011b2eac7db5 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:05.471500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf590f3-efb2-42f3-be34-791a381ae33e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:05.484229 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9b65f-b633-4c89-8369-169ef96b959e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:05.485503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9b65f-b633-4c89-8369-169ef96b959e admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:05.486230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260aeb7d-1655-4e25-abc3-e96bb5482311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:05.487157 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aeb7d-1655-4e25-abc3-e96bb5482311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:05.487440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aeb7d-1655-4e25-abc3-e96bb5482311 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:05.487713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aeb7d-1655-4e25-abc3-e96bb5482311 admin admin] PATH_INFO: `/v3/users/a9a3080f9aef4323952eed19ef9bb074` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:05.487995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9b65f-b633-4c89-8369-169ef96b959e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:05.488399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9b65f-b633-4c89-8369-169ef96b959e admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:05.489252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9b65f-b633-4c89-8369-169ef96b959e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:05.493122 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9b65f-b633-4c89-8369-169ef96b959e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:05.495974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260aeb7d-1655-4e25-abc3-e96bb5482311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a3080f9aef4323952eed19ef9bb074)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:05.497949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260aeb7d-1655-4e25-abc3-e96bb5482311 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:05.542143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1157/2321] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:57:05.543095 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9b65f-b633-4c89-8369-169ef96b959e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:05.587889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9b65f-b633-4c89-8369-169ef96b959e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:05.604567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d3ccf-7770-482e-b631-a990897f33c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:05.605518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d3ccf-7770-482e-b631-a990897f33c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:05.605809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d3ccf-7770-482e-b631-a990897f33c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:05.606092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d3ccf-7770-482e-b631-a990897f33c5 admin admin] PATH_INFO: `/v3/projects/f268207f2a704b56a8499785e70781f6/users/6d31a42762144f96b80768df4a1d79b1/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:05.640590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-260aeb7d-1655-4e25-abc3-e96bb5482311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a3080f9aef4323952eed19ef9bb074'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:05.652927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-260aeb7d-1655-4e25-abc3-e96bb5482311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a3080f9aef4323952eed19ef9bb074'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:05.660627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d3ccf-7770-482e-b631-a990897f33c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f268207f2a704b56a8499785e70781f6, user_id=6d31a42762144f96b80768df4a1d79b1, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:05.663601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d3ccf-7770-482e-b631-a990897f33c5 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:05.668401 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-260aeb7d-1655-4e25-abc3-e96bb5482311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a3080f9aef4323952eed19ef9bb074'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:05.669381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-260aeb7d-1655-4e25-abc3-e96bb5482311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a3080f9aef4323952eed19ef9bb074'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:05.670184 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-260aeb7d-1655-4e25-abc3-e96bb5482311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a3080f9aef4323952eed19ef9bb074'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:05.705576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1165/2322] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:57:05 2022] DELETE /identity/v3/users/a9a3080f9aef4323952eed19ef9bb074 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:57:05.707836 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1158/2323] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:57:05 2022] PUT /identity/v3/projects/f268207f2a704b56a8499785e70781f6/users/6d31a42762144f96b80768df4a1d79b1/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:57:05.720697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb2411-6e5e-4d04-a6bb-211c3c486d3f None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:05.721035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb2411-6e5e-4d04-a6bb-211c3c486d3f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:05.721369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb2411-6e5e-4d04-a6bb-211c3c486d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:05.730122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:05.799008 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-92eb2411-6e5e-4d04-a6bb-211c3c486d3f None None] MFA Rules not processed for user `6d31a42762144f96b80768df4a1d79b1`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:05.800330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-92eb2411-6e5e-4d04-a6bb-211c3c486d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:06.025662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1166/2324] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:57:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:06.213128 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d3ccf-7770-482e-b631-a990897f33c5 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:06.215202 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-363d3ccf-7770-482e-b631-a990897f33c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:06.401324 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d3ccf-7770-482e-b631-a990897f33c5 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:06.452922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:06.555935 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3b17c-f182-4602-ae0e-55defce8dd07 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '666ade4ce1fe447487998d086779389b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:06.556766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3b17c-f182-4602-ae0e-55defce8dd07 service glance] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:06.557161 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3b17c-f182-4602-ae0e-55defce8dd07 service glance] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:06.557546 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3b17c-f182-4602-ae0e-55defce8dd07 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:06.588799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3b17c-f182-4602-ae0e-55defce8dd07 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:06.590502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3b17c-f182-4602-ae0e-55defce8dd07 service glance] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:06.612571 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1159/2325] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:57:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:57:07.071423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef3f2d-87db-4f96-9024-5e113d7688a8 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:07.071955 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef3f2d-87db-4f96-9024-5e113d7688a8 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:07.071955 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef3f2d-87db-4f96-9024-5e113d7688a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:07.115846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-29ef3f2d-87db-4f96-9024-5e113d7688a8 None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:07.123131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-29ef3f2d-87db-4f96-9024-5e113d7688a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:07.178914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3b17c-f182-4602-ae0e-55defce8dd07 service glance] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:07.189815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3ca3b17c-f182-4602-ae0e-55defce8dd07 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:07.291316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1167/2326] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 13:57:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:07.321357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef3f2d-87db-4f96-9024-5e113d7688a8 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:07.323663 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-29ef3f2d-87db-4f96-9024-5e113d7688a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:07.509808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3b17c-f182-4602-ae0e-55defce8dd07 service glance] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:07.523564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:07.524545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:07.524840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:07.525695 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd admin admin] PATH_INFO: `/v3/projects/4fe46ffac52d412daf3663e2ab1273de` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:07.529538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fe46ffac52d412daf3663e2ab1273de)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:07.532422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:07.588465 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef3f2d-87db-4f96-9024-5e113d7688a8 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:07.622294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c4b2e-7631-4fb7-b4e9-18852c40facd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:07.624796 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c4b2e-7631-4fb7-b4e9-18852c40facd service swift] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:07.629117 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c4b2e-7631-4fb7-b4e9-18852c40facd service swift] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:07.629778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c4b2e-7631-4fb7-b4e9-18852c40facd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:07.658772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c4b2e-7631-4fb7-b4e9-18852c40facd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:07.661891 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c4b2e-7631-4fb7-b4e9-18852c40facd service swift] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:07.680836 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fe46ffac52d412daf3663e2ab1273de', 'request_id': 'req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:07.699155 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fe46ffac52d412daf3663e2ab1273de', 'request_id': 'req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:07.701298 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1168/2327] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 13:57:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:57:07.809893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fe46ffac52d412daf3663e2ab1273de', 'request_id': 'req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:07.851698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd admin admin] The token cache is being invalidate because project 4fe46ffac52d412daf3663e2ab1273de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:57:07.853150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:07.854413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1a2fa57a-11c7-457c-a10a-6ad33e11dcfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:07.866806 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b9ff4-1d14-4916-9854-20a66d522ad8 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:07.867949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b9ff4-1d14-4916-9854-20a66d522ad8 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:07.868464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b9ff4-1d14-4916-9854-20a66d522ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:07.871332 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1160/2328] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:57:07 2022] DELETE /identity/v3/projects/4fe46ffac52d412daf3663e2ab1273de => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:57:07.898537 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-461b9ff4-1d14-4916-9854-20a66d522ad8 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:07.900206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-461b9ff4-1d14-4916-9854-20a66d522ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:07.959487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:08.089826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094f5e8-8210-4da0-899a-3bd2bc64ddcc None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:08.090491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094f5e8-8210-4da0-899a-3bd2bc64ddcc None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:08.091110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094f5e8-8210-4da0-899a-3bd2bc64ddcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:08.103107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1169/2329] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:08.164503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-b094f5e8-8210-4da0-899a-3bd2bc64ddcc None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:08.170507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b094f5e8-8210-4da0-899a-3bd2bc64ddcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:08.206957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b9ff4-1d14-4916-9854-20a66d522ad8 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:08.207678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1161/2330] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:08.219410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b9ff4-1d14-4916-9854-20a66d522ad8 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:08.239908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089f1a8-b251-4767-8e74-c637836a9e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:08.241469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089f1a8-b251-4767-8e74-c637836a9e7b admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:08.241886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089f1a8-b251-4767-8e74-c637836a9e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:08.242424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089f1a8-b251-4767-8e74-c637836a9e7b admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:08.243410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089f1a8-b251-4767-8e74-c637836a9e7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:57:08.243948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089f1a8-b251-4767-8e74-c637836a9e7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:08.248532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089f1a8-b251-4767-8e74-c637836a9e7b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:08.299740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094f5e8-8210-4da0-899a-3bd2bc64ddcc None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:08.306569 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1170/2331] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:57:08.315212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094f5e8-8210-4da0-899a-3bd2bc64ddcc None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:08.335793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc6857-9845-469d-b930-5cf5d8fa4464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:08.337547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc6857-9845-469d-b930-5cf5d8fa4464 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:08.341433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc6857-9845-469d-b930-5cf5d8fa4464 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:08.348974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc6857-9845-469d-b930-5cf5d8fa4464 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:08.351484 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc6857-9845-469d-b930-5cf5d8fa4464 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:08.357278 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc6857-9845-469d-b930-5cf5d8fa4464 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:08.472928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089f1a8-b251-4767-8e74-c637836a9e7b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:08.473516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1162/2332] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:57:08 2022] POST /identity/v3/projects => generated 399 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:57:08.498388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089f1a8-b251-4767-8e74-c637836a9e7b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:08.633919 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135f707-1ea2-413f-9755-e83a3e22b49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:08.634853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135f707-1ea2-413f-9755-e83a3e22b49c admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:08.635270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135f707-1ea2-413f-9755-e83a3e22b49c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:08.635667 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135f707-1ea2-413f-9755-e83a3e22b49c admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:08.636674 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135f707-1ea2-413f-9755-e83a3e22b49c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:08.647764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135f707-1ea2-413f-9755-e83a3e22b49c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:08.715031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc6857-9845-469d-b930-5cf5d8fa4464 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:08.729173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1171/2333] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:57:08 2022] POST /identity/v3/users => generated 300 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:57:08.734084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc6857-9845-469d-b930-5cf5d8fa4464 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:08.745667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4060d-ea21-4a96-a947-62c4c9738e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:08.746370 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4060d-ea21-4a96-a947-62c4c9738e04 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:08.749494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4060d-ea21-4a96-a947-62c4c9738e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:08.749825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4060d-ea21-4a96-a947-62c4c9738e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:08.751524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4060d-ea21-4a96-a947-62c4c9738e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:08.755506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4060d-ea21-4a96-a947-62c4c9738e04 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:08.778631 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1163/2334] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:57:08.783527 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135f707-1ea2-413f-9755-e83a3e22b49c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:08.797069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135f707-1ea2-413f-9755-e83a3e22b49c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:08.813625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad7f44-3114-4919-955b-fd5dd9b470d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:08.814471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad7f44-3114-4919-955b-fd5dd9b470d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:08.814767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad7f44-3114-4919-955b-fd5dd9b470d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:08.815081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad7f44-3114-4919-955b-fd5dd9b470d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:08.815835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad7f44-3114-4919-955b-fd5dd9b470d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:08.818833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad7f44-3114-4919-955b-fd5dd9b470d5 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:08.842009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4060d-ea21-4a96-a947-62c4c9738e04 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:08.855359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1172/2335] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:57:08.859468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4060d-ea21-4a96-a947-62c4c9738e04 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:08.869888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d844d-c65f-4ede-bb20-4d5d22bfd6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:08.870803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d844d-c65f-4ede-bb20-4d5d22bfd6fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:08.871116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d844d-c65f-4ede-bb20-4d5d22bfd6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:08.871396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d844d-c65f-4ede-bb20-4d5d22bfd6fe admin admin] PATH_INFO: `/v3/projects/100a9490a1da4fbe9602085539caf60d/users/29aeb00d08c645339aa48876797c4a98/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:08.921000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d844d-c65f-4ede-bb20-4d5d22bfd6fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=100a9490a1da4fbe9602085539caf60d, user_id=29aeb00d08c645339aa48876797c4a98, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:08.932017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d844d-c65f-4ede-bb20-4d5d22bfd6fe admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:08.940003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:08.978383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1164/2336] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:57:08 2022] PUT /identity/v3/projects/100a9490a1da4fbe9602085539caf60d/users/29aeb00d08c645339aa48876797c4a98/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:57:08.993531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc70b55-6a33-4894-9c4f-f36726857dd9 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:08.993908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc70b55-6a33-4894-9c4f-f36726857dd9 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:08.994089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc70b55-6a33-4894-9c4f-f36726857dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:09.052350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-3cc70b55-6a33-4894-9c4f-f36726857dd9 None None] MFA Rules not processed for user `29aeb00d08c645339aa48876797c4a98`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:09.054192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3cc70b55-6a33-4894-9c4f-f36726857dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:09.282696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d844d-c65f-4ede-bb20-4d5d22bfd6fe admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:09.283718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1173/2337] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:57:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:09.284653 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b55d844d-c65f-4ede-bb20-4d5d22bfd6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:09.505125 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d844d-c65f-4ede-bb20-4d5d22bfd6fe admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:09.528884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f713fa9-b853-4af8-ad1f-1e71ed16116d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:09.529564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f713fa9-b853-4af8-ad1f-1e71ed16116d service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:09.529846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f713fa9-b853-4af8-ad1f-1e71ed16116d service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:09.530124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f713fa9-b853-4af8-ad1f-1e71ed16116d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:09.556348 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f713fa9-b853-4af8-ad1f-1e71ed16116d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:09.559838 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f713fa9-b853-4af8-ad1f-1e71ed16116d service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:09.596753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1165/2338] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:57:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:57:09.671125 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2fa5a-2c33-406c-a1cc-e0bae521b01d None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:09.671914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2fa5a-2c33-406c-a1cc-e0bae521b01d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:09.671914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2fa5a-2c33-406c-a1cc-e0bae521b01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:09.722502 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-84b2fa5a-2c33-406c-a1cc-e0bae521b01d None None] MFA Rules not processed for user `d5fabcc754684ded96c800be75777553`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:09.724586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-84b2fa5a-2c33-406c-a1cc-e0bae521b01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:09.763219 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:09.942353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1174/2339] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:57:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:09.946161 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f713fa9-b853-4af8-ad1f-1e71ed16116d service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:09.947643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-0f713fa9-b853-4af8-ad1f-1e71ed16116d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:10.212836 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f713fa9-b853-4af8-ad1f-1e71ed16116d service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:10.223251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bdb03-03f0-40a3-ba72-e3279785dfb6 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:10.223831 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bdb03-03f0-40a3-ba72-e3279785dfb6 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:10.224329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bdb03-03f0-40a3-ba72-e3279785dfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:10.235913 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5172a3-6379-462e-b7bd-2ee2156de9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:10.239973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5172a3-6379-462e-b7bd-2ee2156de9e9 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:10.242350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5172a3-6379-462e-b7bd-2ee2156de9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:10.242691 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5172a3-6379-462e-b7bd-2ee2156de9e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:10.243510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5172a3-6379-462e-b7bd-2ee2156de9e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:10.253356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5172a3-6379-462e-b7bd-2ee2156de9e9 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:10.268260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-943bdb03-03f0-40a3-ba72-e3279785dfb6 None None] MFA Rules not processed for user `a1274435c6774faa9eeeac6726b52b47`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:10.283492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-943bdb03-03f0-40a3-ba72-e3279785dfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:10.339891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1166/2340] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:57:09 2022] POST /identity/v3/projects => generated 371 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:57:10.344179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5172a3-6379-462e-b7bd-2ee2156de9e9 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:10.498058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5172a3-6379-462e-b7bd-2ee2156de9e9 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:10.513713 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5809-4b41-474f-a3ba-88270f0ec968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:10.514409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5809-4b41-474f-a3ba-88270f0ec968 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:10.514696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5809-4b41-474f-a3ba-88270f0ec968 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:10.515026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5809-4b41-474f-a3ba-88270f0ec968 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:10.516116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c5809-4b41-474f-a3ba-88270f0ec968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:10.518418 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c5809-4b41-474f-a3ba-88270f0ec968 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:10.541585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1175/2341] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:57:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:10.593230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1167/2342] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:57:10 2022] POST /identity/v3/users => generated 292 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:57:10.596605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943bdb03-03f0-40a3-ba72-e3279785dfb6 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:10.629954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943bdb03-03f0-40a3-ba72-e3279785dfb6 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:10.649000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa5305-a299-4893-9c45-c23cd386493b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:10.649718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5305-a299-4893-9c45-c23cd386493b admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:10.650094 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5305-a299-4893-9c45-c23cd386493b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:10.659336 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5305-a299-4893-9c45-c23cd386493b admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:10.660813 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa5305-a299-4893-9c45-c23cd386493b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:10.667847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa5305-a299-4893-9c45-c23cd386493b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:10.725559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1176/2343] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:57:10.726847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5809-4b41-474f-a3ba-88270f0ec968 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:10.738040 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5809-4b41-474f-a3ba-88270f0ec968 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:10.759945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89967833-9956-4eb9-9e16-a671182eb695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:10.761554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89967833-9956-4eb9-9e16-a671182eb695 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:10.762070 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89967833-9956-4eb9-9e16-a671182eb695 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:10.762758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89967833-9956-4eb9-9e16-a671182eb695 admin admin] PATH_INFO: `/v3/projects/f70c3715c82e42958ab6bb02373acc2c/users/b0d9ebf3b3d046e79f6032573cc0a2bf/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:10.842495 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:10.874096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89967833-9956-4eb9-9e16-a671182eb695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70c3715c82e42958ab6bb02373acc2c, user_id=b0d9ebf3b3d046e79f6032573cc0a2bf, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:10.887965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89967833-9956-4eb9-9e16-a671182eb695 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:10.932542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1168/2344] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:57:10 2022] PUT /identity/v3/projects/f70c3715c82e42958ab6bb02373acc2c/users/b0d9ebf3b3d046e79f6032573cc0a2bf/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:57:10.936344 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa5305-a299-4893-9c45-c23cd386493b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:10.952153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa5305-a299-4893-9c45-c23cd386493b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:11.121285 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36313f-a5e8-4a57-809f-1f9c4eb9310e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:11.123327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36313f-a5e8-4a57-809f-1f9c4eb9310e admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:11.125370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36313f-a5e8-4a57-809f-1f9c4eb9310e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:11.127281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36313f-a5e8-4a57-809f-1f9c4eb9310e admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:11.128885 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36313f-a5e8-4a57-809f-1f9c4eb9310e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:11.133587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36313f-a5e8-4a57-809f-1f9c4eb9310e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:11.174218 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1177/2345] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:57:11.175564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89967833-9956-4eb9-9e16-a671182eb695 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:11.196765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89967833-9956-4eb9-9e16-a671182eb695 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:11.211757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d058d-f042-4798-8b67-c706be47f805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:11.221361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d058d-f042-4798-8b67-c706be47f805 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:11.221678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d058d-f042-4798-8b67-c706be47f805 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:11.221955 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d058d-f042-4798-8b67-c706be47f805 admin admin] PATH_INFO: `/v3/projects/f70c3715c82e42958ab6bb02373acc2c/users/b0d9ebf3b3d046e79f6032573cc0a2bf/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:11.230647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d058d-f042-4798-8b67-c706be47f805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70c3715c82e42958ab6bb02373acc2c, user_id=b0d9ebf3b3d046e79f6032573cc0a2bf, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:11.240618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d058d-f042-4798-8b67-c706be47f805 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:11.294134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1169/2346] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:57:11 2022] PUT /identity/v3/projects/f70c3715c82e42958ab6bb02373acc2c/users/b0d9ebf3b3d046e79f6032573cc0a2bf/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:57:19.426371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36313f-a5e8-4a57-809f-1f9c4eb9310e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:19.444368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36313f-a5e8-4a57-809f-1f9c4eb9310e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:19.572882 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:19.607767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:19.608672 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:19.608960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:19.609795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] PATH_INFO: `/v3/users/29aeb00d08c645339aa48876797c4a98` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:19.611153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29aeb00d08c645339aa48876797c4a98)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:19.613143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:19.776312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29aeb00d08c645339aa48876797c4a98'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:19.804080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29aeb00d08c645339aa48876797c4a98'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:19.819075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29aeb00d08c645339aa48876797c4a98'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:19.820131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29aeb00d08c645339aa48876797c4a98'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:19.827375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29aeb00d08c645339aa48876797c4a98'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:19.898832 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1178/2347] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:57:19 2022] DELETE /identity/v3/users/29aeb00d08c645339aa48876797c4a98 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:57:19.903160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841c584-dab0-4d38-bb08-56ccdd70c71d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:19.904084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841c584-dab0-4d38-bb08-56ccdd70c71d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:19.904709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841c584-dab0-4d38-bb08-56ccdd70c71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:19.952203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-2841c584-dab0-4d38-bb08-56ccdd70c71d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:19.954369 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2841c584-dab0-4d38-bb08-56ccdd70c71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:20.147384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1170/2348] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:20.161076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:20.162557 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:20.324844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6100b7-6c7c-4ad2-a462-260b1626f972 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:20.341836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f001d32-888b-43d1-a423-6343566d5885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:20.343060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f001d32-888b-43d1-a423-6343566d5885 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:20.343710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f001d32-888b-43d1-a423-6343566d5885 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:20.344214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f001d32-888b-43d1-a423-6343566d5885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:20.377402 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f001d32-888b-43d1-a423-6343566d5885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:20.381424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f001d32-888b-43d1-a423-6343566d5885 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:20.418574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1179/2349] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:57:20.848025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afbaac-b7bf-4113-ae44-93740f7bf38c None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:20.852018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afbaac-b7bf-4113-ae44-93740f7bf38c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:20.852657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afbaac-b7bf-4113-ae44-93740f7bf38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:20.917506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-03afbaac-b7bf-4113-ae44-93740f7bf38c None None] MFA Rules not processed for user `b0d9ebf3b3d046e79f6032573cc0a2bf`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:20.923919 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-03afbaac-b7bf-4113-ae44-93740f7bf38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:21.094904 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1171/2350] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:57:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:21.105594 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f001d32-888b-43d1-a423-6343566d5885 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:21.107215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2f001d32-888b-43d1-a423-6343566d5885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:21.247182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:21.307805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f001d32-888b-43d1-a423-6343566d5885 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:21.316255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea125dfd-579c-4241-a67c-373f7b554603 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:21.329858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea125dfd-579c-4241-a67c-373f7b554603 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:21.330356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea125dfd-579c-4241-a67c-373f7b554603 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:21.330925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea125dfd-579c-4241-a67c-373f7b554603 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:21.336078 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ea125dfd-579c-4241-a67c-373f7b554603 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:21.429474 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afbaac-b7bf-4113-ae44-93740f7bf38c None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:21.432489 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-03afbaac-b7bf-4113-ae44-93740f7bf38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:21.545779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afbaac-b7bf-4113-ae44-93740f7bf38c None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:21.575246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:21.581641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:21.582266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:21.582354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] PATH_INFO: `/v3/projects/100a9490a1da4fbe9602085539caf60d` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:21.588254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=100a9490a1da4fbe9602085539caf60d)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:21.602822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:21.627375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea125dfd-579c-4241-a67c-373f7b554603 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:21.630398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea125dfd-579c-4241-a67c-373f7b554603 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:21.644778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1180/2351] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:57:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:57:21.743747 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '100a9490a1da4fbe9602085539caf60d', 'request_id': 'req-42c463a6-676b-438d-9082-dfb380d7b760'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:21.772325 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '100a9490a1da4fbe9602085539caf60d', 'request_id': 'req-42c463a6-676b-438d-9082-dfb380d7b760'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:21.812758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:21.836687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '100a9490a1da4fbe9602085539caf60d', 'request_id': 'req-42c463a6-676b-438d-9082-dfb380d7b760'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:21.866886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] The token cache is being invalidate because project 100a9490a1da4fbe9602085539caf60d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:57:21.874239 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:21.874239 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:21.876254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1172/2352] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:57:21 2022] DELETE /identity/v3/projects/100a9490a1da4fbe9602085539caf60d => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:57:22.276691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba490a8-0c5e-4560-8784-0f524f49e1f3 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:22.277157 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba490a8-0c5e-4560-8784-0f524f49e1f3 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:22.277453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba490a8-0c5e-4560-8784-0f524f49e1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:22.348538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-2ba490a8-0c5e-4560-8784-0f524f49e1f3 None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:22.358223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2ba490a8-0c5e-4560-8784-0f524f49e1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:22.578833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1181/2353] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 13:57:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:22.593179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:22.595421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:22.740271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c463a6-676b-438d-9082-dfb380d7b760 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:22.759409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a53919-918e-4ced-b4c2-af2b74390037 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:22.760094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a53919-918e-4ced-b4c2-af2b74390037 service swift] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:22.760380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a53919-918e-4ced-b4c2-af2b74390037 service swift] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:22.760659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a53919-918e-4ced-b4c2-af2b74390037 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:22.785480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a53919-918e-4ced-b4c2-af2b74390037 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:22.785998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a53919-918e-4ced-b4c2-af2b74390037 service swift] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:22.804858 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1173/2354] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 13:57:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:57:24.876927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866fcab-ee4c-44cb-90a1-3a671dc53ddb None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:24.877385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866fcab-ee4c-44cb-90a1-3a671dc53ddb None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:24.877544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866fcab-ee4c-44cb-90a1-3a671dc53ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:24.918613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-9866fcab-ee4c-44cb-90a1-3a671dc53ddb None None] MFA Rules not processed for user `d5fabcc754684ded96c800be75777553`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:24.920157 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9866fcab-ee4c-44cb-90a1-3a671dc53ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:25.069136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1182/2355] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:57:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:32.799748 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=36 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:35.044950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=85 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:41.871348 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a53919-918e-4ced-b4c2-af2b74390037 service swift] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:41.872826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c7a53919-918e-4ced-b4c2-af2b74390037 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:41.963974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a53919-918e-4ced-b4c2-af2b74390037 service swift] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:41.969069 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972c168-b4c3-4902-b120-63a9867317db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b105f8163ce84a9ba725e6b04d348446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:41.969714 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972c168-b4c3-4902-b120-63a9867317db service placement] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:41.969998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972c168-b4c3-4902-b120-63a9867317db service placement] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:41.970274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972c168-b4c3-4902-b120-63a9867317db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:41.972613 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c972c168-b4c3-4902-b120-63a9867317db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:41.984232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972c168-b4c3-4902-b120-63a9867317db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:41.985803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972c168-b4c3-4902-b120-63a9867317db service placement] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:42.003869 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1174/2356] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:57:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:57:43.184737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123b89a-3c60-4906-b90d-f08fba27648f None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:43.185332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123b89a-3c60-4906-b90d-f08fba27648f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:43.185432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123b89a-3c60-4906-b90d-f08fba27648f None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:43.186892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1183/2357] 10.222.0.52 () {54 vars in 921 bytes} [Thu May 19 13:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:57:43.193303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7400e-dfc9-4544-b1a7-a3e81db04ece None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:43.193759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7400e-dfc9-4544-b1a7-a3e81db04ece None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:43.193985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7400e-dfc9-4544-b1a7-a3e81db04ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:43.235651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-84e7400e-dfc9-4544-b1a7-a3e81db04ece None None] MFA Rules not processed for user `4f6093d0c1fc41d98c58bace81e3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:43.237034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-84e7400e-dfc9-4544-b1a7-a3e81db04ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:43.306956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1175/2358] 10.222.0.52 () {58 vars in 1018 bytes} [Thu May 19 13:57:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:43.320059 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123b89a-3c60-4906-b90d-f08fba27648f None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:43.322235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c123b89a-3c60-4906-b90d-f08fba27648f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:43.386538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123b89a-3c60-4906-b90d-f08fba27648f None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:43.392981 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c5d38-6a25-423c-bd03-8614d6ae2b0a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '666ade4ce1fe447487998d086779389b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:43.394312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c5d38-6a25-423c-bd03-8614d6ae2b0a service glance] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:43.394596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c5d38-6a25-423c-bd03-8614d6ae2b0a service glance] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:43.395508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c5d38-6a25-423c-bd03-8614d6ae2b0a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:43.406086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c5d38-6a25-423c-bd03-8614d6ae2b0a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:43.407549 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c5d38-6a25-423c-bd03-8614d6ae2b0a service glance] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:43.418715 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1184/2359] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:57:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:57:43.483303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506dc16-3aa1-4689-8fbb-6cefc8231e04 None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:43.483596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506dc16-3aa1-4689-8fbb-6cefc8231e04 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:43.483875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506dc16-3aa1-4689-8fbb-6cefc8231e04 None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:43.485238 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1176/2360] 10.222.0.52 () {54 vars in 921 bytes} [Thu May 19 13:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:57:43.490665 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640b4bb-fe31-43ff-af21-ad7feceb16b3 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:43.491011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640b4bb-fe31-43ff-af21-ad7feceb16b3 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:43.491307 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640b4bb-fe31-43ff-af21-ad7feceb16b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:43.509591 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f640b4bb-fe31-43ff-af21-ad7feceb16b3 None None] MFA Rules not processed for user `f18f635bff0142409e90da66547927f1`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:43.510799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f640b4bb-fe31-43ff-af21-ad7feceb16b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:43.562406 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1185/2361] 10.222.0.52 () {58 vars in 1018 bytes} [Thu May 19 13:57:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:43.574049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506dc16-3aa1-4689-8fbb-6cefc8231e04 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:43.576120 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9506dc16-3aa1-4689-8fbb-6cefc8231e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:43.589900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506dc16-3aa1-4689-8fbb-6cefc8231e04 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:43.593693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfc2fc-c897-4957-9aca-888ae1bc9a93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:43.594403 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfc2fc-c897-4957-9aca-888ae1bc9a93 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:43.594783 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfc2fc-c897-4957-9aca-888ae1bc9a93 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:43.595100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfc2fc-c897-4957-9aca-888ae1bc9a93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:43.604698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfc2fc-c897-4957-9aca-888ae1bc9a93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:43.606299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfc2fc-c897-4957-9aca-888ae1bc9a93 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:43.614927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1177/2362] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:57:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:57:43.704383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd06bb3-c2a0-4581-9614-39920192041e None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:43.704883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd06bb3-c2a0-4581-9614-39920192041e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:43.705247 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd06bb3-c2a0-4581-9614-39920192041e None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:43.707829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1186/2363] 10.222.0.52 () {54 vars in 921 bytes} [Thu May 19 13:57:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:57:43.713809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c1040-b54a-4a4e-a3d8-105ef3e67f9d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:43.714624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c1040-b54a-4a4e-a3d8-105ef3e67f9d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:43.715197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c1040-b54a-4a4e-a3d8-105ef3e67f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:43.740464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-dc2c1040-b54a-4a4e-a3d8-105ef3e67f9d None None] MFA Rules not processed for user `4f6093d0c1fc41d98c58bace81e3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:43.741618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-dc2c1040-b54a-4a4e-a3d8-105ef3e67f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:43.751616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1178/2364] 10.222.0.52 () {58 vars in 1018 bytes} [Thu May 19 13:57:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:43.763475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd06bb3-c2a0-4581-9614-39920192041e None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:43.765264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-afd06bb3-c2a0-4581-9614-39920192041e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:43.783580 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd06bb3-c2a0-4581-9614-39920192041e None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:43.790825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff5799-02d7-4199-a1af-fb1bfe895b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:43.791868 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff5799-02d7-4199-a1af-fb1bfe895b99 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:43.792166 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff5799-02d7-4199-a1af-fb1bfe895b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:43.792520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff5799-02d7-4199-a1af-fb1bfe895b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:43.805174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff5799-02d7-4199-a1af-fb1bfe895b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:43.806895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff5799-02d7-4199-a1af-fb1bfe895b99 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:43.818794 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1187/2365] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:57:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:57:47.031901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c1040-b54a-4a4e-a3d8-105ef3e67f9d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:47.034166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-dc2c1040-b54a-4a4e-a3d8-105ef3e67f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:47.066807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:47.104681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c1040-b54a-4a4e-a3d8-105ef3e67f9d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:47.109283 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:47.110153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:47.110441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:47.110717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] PATH_INFO: `/v3/users/6d31a42762144f96b80768df4a1d79b1` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:47.113568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d31a42762144f96b80768df4a1d79b1)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:47.113996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:47.170202 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d31a42762144f96b80768df4a1d79b1'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:47.179346 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d31a42762144f96b80768df4a1d79b1'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:47.180172 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d31a42762144f96b80768df4a1d79b1'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:47.191890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d31a42762144f96b80768df4a1d79b1'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:47.213420 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d31a42762144f96b80768df4a1d79b1'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:47.219996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1179/2366] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:57:47 2022] DELETE /identity/v3/users/6d31a42762144f96b80768df4a1d79b1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:57:47.225829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb62890-a006-48f4-b160-c7a9bdc831ee None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:47.226346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb62890-a006-48f4-b160-c7a9bdc831ee None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:47.226546 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb62890-a006-48f4-b160-c7a9bdc831ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:47.246841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-8bb62890-a006-48f4-b160-c7a9bdc831ee None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:47.248238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8bb62890-a006-48f4-b160-c7a9bdc831ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:47.314327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1188/2367] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:47.325171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:47.326997 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:47.389471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adecaef-8bc2-4a74-a396-81368a72530b admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:47.394780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d4aef-76aa-4613-a091-656c6493721c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:47.395900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d4aef-76aa-4613-a091-656c6493721c service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:47.396192 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d4aef-76aa-4613-a091-656c6493721c service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:47.396480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d4aef-76aa-4613-a091-656c6493721c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:47.408064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d4aef-76aa-4613-a091-656c6493721c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:47.410364 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d4aef-76aa-4613-a091-656c6493721c service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:47.420867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1180/2368] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:57:47.891016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb62890-a006-48f4-b160-c7a9bdc831ee None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:47.892630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8bb62890-a006-48f4-b160-c7a9bdc831ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:47.905635 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb62890-a006-48f4-b160-c7a9bdc831ee None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:47.914307 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:47.917291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:47.917587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:47.917873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] PATH_INFO: `/v3/projects/f268207f2a704b56a8499785e70781f6` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:47.920632 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f268207f2a704b56a8499785e70781f6)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:47.922659 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:47.972377 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:47.988920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f268207f2a704b56a8499785e70781f6', 'request_id': 'req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:47.997438 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f268207f2a704b56a8499785e70781f6', 'request_id': 'req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:48.007678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f268207f2a704b56a8499785e70781f6', 'request_id': 'req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:48.023220 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] The token cache is being invalidate because project f268207f2a704b56a8499785e70781f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:57:48.032853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:48.033737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:57:48.051108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1189/2369] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:57:47 2022] DELETE /identity/v3/projects/f268207f2a704b56a8499785e70781f6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:57:48.078382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77056782-1410-4685-9567-b698f0cc9903 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:48.078996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77056782-1410-4685-9567-b698f0cc9903 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:48.078996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77056782-1410-4685-9567-b698f0cc9903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:48.103955 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-77056782-1410-4685-9567-b698f0cc9903 None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:57:48.105209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-77056782-1410-4685-9567-b698f0cc9903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:48.174392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:48.198536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1181/2370] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 13:57:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:57:48.227435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:57:48.229013 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:57:48.355484 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8f0cd-f923-4eb0-a1ee-24c0d3bb32fc admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:57:48.363277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4374ac-a4b1-465c-adaa-3691757480eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:57:48.364024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4374ac-a4b1-465c-adaa-3691757480eb service swift] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:57:48.364421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4374ac-a4b1-465c-adaa-3691757480eb service swift] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:57:48.364795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4374ac-a4b1-465c-adaa-3691757480eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:57:48.376676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4374ac-a4b1-465c-adaa-3691757480eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:57:48.380330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4374ac-a4b1-465c-adaa-3691757480eb service swift] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:57:48.392696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1190/2371] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 13:57:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:57:58.217047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:57:58.392914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:DELETE=5,keystone:SELECT=43,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:58:06.959369 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77056782-1410-4685-9567-b698f0cc9903 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:58:06.961006 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-77056782-1410-4685-9567-b698f0cc9903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:58:07.019445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77056782-1410-4685-9567-b698f0cc9903 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:58:07.023420 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6828179-003c-49d2-955f-3cd89fe40c03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:58:07.024077 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6828179-003c-49d2-955f-3cd89fe40c03 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:58:07.024361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6828179-003c-49d2-955f-3cd89fe40c03 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:58:07.024681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6828179-003c-49d2-955f-3cd89fe40c03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:58:07.026512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b6828179-003c-49d2-955f-3cd89fe40c03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:58:07.097533 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6828179-003c-49d2-955f-3cd89fe40c03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:58:07.099385 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6828179-003c-49d2-955f-3cd89fe40c03 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:58:07.109392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1182/2372] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:58:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:58:12.188109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5aa19-f0e2-498a-8f61-ddfa0c93fe42 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:58:12.188645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5aa19-f0e2-498a-8f61-ddfa0c93fe42 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:58:12.188645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5aa19-f0e2-498a-8f61-ddfa0c93fe42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:58:12.209476 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-9fd5aa19-f0e2-498a-8f61-ddfa0c93fe42 None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:58:12.211412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9fd5aa19-f0e2-498a-8f61-ddfa0c93fe42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:58:12.219809 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1191/2373] 10.222.0.52 () {58 vars in 967 bytes} [Thu May 19 13:58:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:58:12.238370 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6828179-003c-49d2-955f-3cd89fe40c03 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:58:12.254241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6828179-003c-49d2-955f-3cd89fe40c03 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:58:12.259572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928da2fd-7c20-42ec-a300-bef585e41826 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:58:12.260217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da2fd-7c20-42ec-a300-bef585e41826 service swift] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:58:12.260528 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da2fd-7c20-42ec-a300-bef585e41826 service swift] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:58:12.260762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da2fd-7c20-42ec-a300-bef585e41826 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:58:12.269760 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928da2fd-7c20-42ec-a300-bef585e41826 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:58:12.271359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928da2fd-7c20-42ec-a300-bef585e41826 service swift] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:58:12.279704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1183/2374] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 13:58:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:58:19.436020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5aa19-f0e2-498a-8f61-ddfa0c93fe42 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:58:19.446100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5aa19-f0e2-498a-8f61-ddfa0c93fe42 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:58:19.449731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831e193-ea10-484f-bf8d-f52f837278d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:58:19.450370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831e193-ea10-484f-bf8d-f52f837278d3 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:58:19.450654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831e193-ea10-484f-bf8d-f52f837278d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:58:19.451070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831e193-ea10-484f-bf8d-f52f837278d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:58:19.452879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1831e193-ea10-484f-bf8d-f52f837278d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:58:19.515894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831e193-ea10-484f-bf8d-f52f837278d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:58:19.517337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831e193-ea10-484f-bf8d-f52f837278d3 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:58:19.530573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1192/2375] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:58:19 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:58:22.276435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=56 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:58:25.796583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4e4f6-abea-4b66-b064-20ebaf9677b7 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:58:25.797225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4e4f6-abea-4b66-b064-20ebaf9677b7 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:58:25.797225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4e4f6-abea-4b66-b064-20ebaf9677b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:58:25.815194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-40d4e4f6-abea-4b66-b064-20ebaf9677b7 None None] MFA Rules not processed for user `698ccea06fb24beea18c679d53f907a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:58:25.816386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-40d4e4f6-abea-4b66-b064-20ebaf9677b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:58:25.872145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1184/2376] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:58:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:58:29.524121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=33 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:58:35.865573 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=25 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:58:41.939509 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55bd51-bf9f-4659-bfa6-1c5b86127ddc None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:58:41.939998 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55bd51-bf9f-4659-bfa6-1c5b86127ddc None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:58:41.940092 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55bd51-bf9f-4659-bfa6-1c5b86127ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:58:41.958230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-7c55bd51-bf9f-4659-bfa6-1c5b86127ddc None None] MFA Rules not processed for user `0cc3e0105f2a472092423852b1f39979`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:58:41.959453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-7c55bd51-bf9f-4659-bfa6-1c5b86127ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:58:42.014314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1193/2377] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:58:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:58:52.007646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=25 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:59:04.449038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c4cb3-9030-49e4-950f-88c79e31d08e None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:04.449580 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c4cb3-9030-49e4-950f-88c79e31d08e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:04.449580 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c4cb3-9030-49e4-950f-88c79e31d08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:04.467468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6e3c4cb3-9030-49e4-950f-88c79e31d08e None None] MFA Rules not processed for user `00b21c3634cc41aca680152740644168`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:59:04.468666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6e3c4cb3-9030-49e4-950f-88c79e31d08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:04.525820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1185/2378] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:59:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:59:14.518458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=25 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:59:15.234776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55bd51-bf9f-4659-bfa6-1c5b86127ddc None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:15.245361 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55bd51-bf9f-4659-bfa6-1c5b86127ddc None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:15.248930 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb858b-4633-403e-bf30-cf1cae1d88eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:15.249563 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb858b-4633-403e-bf30-cf1cae1d88eb service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:15.249848 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb858b-4633-403e-bf30-cf1cae1d88eb service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:15.250128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb858b-4633-403e-bf30-cf1cae1d88eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:15.251913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2abb858b-4633-403e-bf30-cf1cae1d88eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:15.323771 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb858b-4633-403e-bf30-cf1cae1d88eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:15.325217 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb858b-4633-403e-bf30-cf1cae1d88eb service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:15.336597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1194/2379] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:59:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:59:15.537084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c4cb3-9030-49e4-950f-88c79e31d08e None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:15.547183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c4cb3-9030-49e4-950f-88c79e31d08e None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:15.550942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff1983-b4af-4344-9195-d5d4e43595ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:15.551607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff1983-b4af-4344-9195-d5d4e43595ff service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:15.551891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff1983-b4af-4344-9195-d5d4e43595ff service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:15.552165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff1983-b4af-4344-9195-d5d4e43595ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:15.554243 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5eff1983-b4af-4344-9195-d5d4e43595ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:15.567346 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff1983-b4af-4344-9195-d5d4e43595ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:15.568755 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff1983-b4af-4344-9195-d5d4e43595ff service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:15.580558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1186/2380] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:59:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:59:25.331403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=34 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:59:25.522004 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb858b-4633-403e-bf30-cf1cae1d88eb service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:25.523422 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2abb858b-4633-403e-bf30-cf1cae1d88eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:25.575210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=6 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:59:25.579939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb858b-4633-403e-bf30-cf1cae1d88eb service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:25.583658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44abf51-48d1-47e8-92f1-d667affe2d7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:25.584244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44abf51-48d1-47e8-92f1-d667affe2d7e service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:25.584535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44abf51-48d1-47e8-92f1-d667affe2d7e service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:25.584812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44abf51-48d1-47e8-92f1-d667affe2d7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:25.586581 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-b44abf51-48d1-47e8-92f1-d667affe2d7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:25.597741 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44abf51-48d1-47e8-92f1-d667affe2d7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:25.599214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44abf51-48d1-47e8-92f1-d667affe2d7e service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:25.609113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1195/2381] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:59:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:59:35.605038 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=26 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:59:36.506985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff1983-b4af-4344-9195-d5d4e43595ff service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:36.508491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5eff1983-b4af-4344-9195-d5d4e43595ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:36.521673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff1983-b4af-4344-9195-d5d4e43595ff service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:36.525524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c79dfb-eafc-46d3-9ccc-7cb521ac8d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:36.526195 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c79dfb-eafc-46d3-9ccc-7cb521ac8d4c service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:36.526477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c79dfb-eafc-46d3-9ccc-7cb521ac8d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:36.526747 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c79dfb-eafc-46d3-9ccc-7cb521ac8d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:36.528584 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-59c79dfb-eafc-46d3-9ccc-7cb521ac8d4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:36.601830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c79dfb-eafc-46d3-9ccc-7cb521ac8d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:36.603788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c79dfb-eafc-46d3-9ccc-7cb521ac8d4c service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:36.613513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1187/2382] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:59:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:59:37.313886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44abf51-48d1-47e8-92f1-d667affe2d7e service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:37.325082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44abf51-48d1-47e8-92f1-d667affe2d7e service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:37.328981 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb35adc-2e36-479f-9032-16722d922cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:37.329740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb35adc-2e36-479f-9032-16722d922cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:37.330134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb35adc-2e36-479f-9032-16722d922cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:37.330516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb35adc-2e36-479f-9032-16722d922cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:37.332494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ecb35adc-2e36-479f-9032-16722d922cb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:37.344063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb35adc-2e36-479f-9032-16722d922cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:37.345644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb35adc-2e36-479f-9032-16722d922cb3 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:37.356756 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1196/2383] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:59:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:59:40.458295 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c79dfb-eafc-46d3-9ccc-7cb521ac8d4c service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:40.459945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-59c79dfb-eafc-46d3-9ccc-7cb521ac8d4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:40.480492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c79dfb-eafc-46d3-9ccc-7cb521ac8d4c service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:40.485888 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e987e3-5287-41b5-868c-dce349a0627d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:40.486798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e987e3-5287-41b5-868c-dce349a0627d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:40.487151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e987e3-5287-41b5-868c-dce349a0627d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:40.487415 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e987e3-5287-41b5-868c-dce349a0627d admin admin] PATH_INFO: `/v3/users/698ccea06fb24beea18c679d53f907a6` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:40.488749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e987e3-5287-41b5-868c-dce349a0627d admin admin] RBAC: Authorizing `identity:delete_user(user_id=698ccea06fb24beea18c679d53f907a6)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:40.490844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e987e3-5287-41b5-868c-dce349a0627d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:40.551156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-04e987e3-5287-41b5-868c-dce349a0627d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '698ccea06fb24beea18c679d53f907a6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:40.559898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-04e987e3-5287-41b5-868c-dce349a0627d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '698ccea06fb24beea18c679d53f907a6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:40.560664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-04e987e3-5287-41b5-868c-dce349a0627d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '698ccea06fb24beea18c679d53f907a6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:40.568347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-04e987e3-5287-41b5-868c-dce349a0627d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '698ccea06fb24beea18c679d53f907a6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:40.580362 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-04e987e3-5287-41b5-868c-dce349a0627d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '698ccea06fb24beea18c679d53f907a6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:40.586197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1188/2384] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:59:40 2022] DELETE /identity/v3/users/698ccea06fb24beea18c679d53f907a6 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:59:40.762334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb35adc-2e36-479f-9032-16722d922cb3 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:40.780494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb35adc-2e36-479f-9032-16722d922cb3 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:40.837650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e6611-2533-4a5f-b923-428de635e184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:40.838521 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e6611-2533-4a5f-b923-428de635e184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:40.838809 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e6611-2533-4a5f-b923-428de635e184 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:40.839121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e6611-2533-4a5f-b923-428de635e184 admin admin] PATH_INFO: `/v3/projects/28027af4cc0c4ca99035e48aa6237805` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:40.840383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e6611-2533-4a5f-b923-428de635e184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28027af4cc0c4ca99035e48aa6237805)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:40.842306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e6611-2533-4a5f-b923-428de635e184 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:40.891167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-546e6611-2533-4a5f-b923-428de635e184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28027af4cc0c4ca99035e48aa6237805', 'request_id': 'req-546e6611-2533-4a5f-b923-428de635e184'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:40.896626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-546e6611-2533-4a5f-b923-428de635e184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28027af4cc0c4ca99035e48aa6237805', 'request_id': 'req-546e6611-2533-4a5f-b923-428de635e184'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:40.904652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-546e6611-2533-4a5f-b923-428de635e184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28027af4cc0c4ca99035e48aa6237805', 'request_id': 'req-546e6611-2533-4a5f-b923-428de635e184'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:40.915626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-546e6611-2533-4a5f-b923-428de635e184 admin admin] The token cache is being invalidate because project 28027af4cc0c4ca99035e48aa6237805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:59:40.915908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-546e6611-2533-4a5f-b923-428de635e184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:40.916531 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-546e6611-2533-4a5f-b923-428de635e184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:40.918761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1197/2385] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:59:40 2022] DELETE /identity/v3/projects/28027af4cc0c4ca99035e48aa6237805 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:59:40.924402 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd71840-dfea-4f28-a070-c90102069aa9 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:40.924884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd71840-dfea-4f28-a070-c90102069aa9 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:40.925172 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd71840-dfea-4f28-a070-c90102069aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:40.948456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-bcd71840-dfea-4f28-a070-c90102069aa9 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:59:40.949839 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-bcd71840-dfea-4f28-a070-c90102069aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:41.015198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1189/2386] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:59:41.023116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179652f-7f30-4707-a1a9-12abf09eb1b0 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:41.023411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179652f-7f30-4707-a1a9-12abf09eb1b0 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:41.023692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179652f-7f30-4707-a1a9-12abf09eb1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:41.042845 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-f179652f-7f30-4707-a1a9-12abf09eb1b0 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:59:41.044098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f179652f-7f30-4707-a1a9-12abf09eb1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:41.052143 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1198/2387] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:59:41.056917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd71840-dfea-4f28-a070-c90102069aa9 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:41.067100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd71840-dfea-4f28-a070-c90102069aa9 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:41.072919 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615d942-6eb1-4e8b-8d78-311fbce5be75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:41.073580 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615d942-6eb1-4e8b-8d78-311fbce5be75 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:41.073886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615d942-6eb1-4e8b-8d78-311fbce5be75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:41.074164 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615d942-6eb1-4e8b-8d78-311fbce5be75 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:41.075035 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615d942-6eb1-4e8b-8d78-311fbce5be75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:59:41.075501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615d942-6eb1-4e8b-8d78-311fbce5be75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:41.076798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179652f-7f30-4707-a1a9-12abf09eb1b0 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:41.077130 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615d942-6eb1-4e8b-8d78-311fbce5be75 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:41.078464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f179652f-7f30-4707-a1a9-12abf09eb1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:41.079079 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:59:41.096602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1190/2388] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 13:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:59:41.097635 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615d942-6eb1-4e8b-8d78-311fbce5be75 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:41.108236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615d942-6eb1-4e8b-8d78-311fbce5be75 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:41.112880 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d7ca3-e0e9-4c91-a996-9a679961b5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:41.113522 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d7ca3-e0e9-4c91-a996-9a679961b5cf admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:41.113815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d7ca3-e0e9-4c91-a996-9a679961b5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:41.114091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d7ca3-e0e9-4c91-a996-9a679961b5cf admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:41.114772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d7ca3-e0e9-4c91-a996-9a679961b5cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:41.116725 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d7ca3-e0e9-4c91-a996-9a679961b5cf admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:41.147060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1191/2389] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 13:59:41 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:59:41.148167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d7ca3-e0e9-4c91-a996-9a679961b5cf admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:41.159112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179652f-7f30-4707-a1a9-12abf09eb1b0 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:41.163913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:59:41.213749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d7ca3-e0e9-4c91-a996-9a679961b5cf admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:41.218477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee3972-2807-49b8-8277-006740c56ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:41.219174 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee3972-2807-49b8-8277-006740c56ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:41.219460 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee3972-2807-49b8-8277-006740c56ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:41.219737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee3972-2807-49b8-8277-006740c56ec4 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:41.220411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee3972-2807-49b8-8277-006740c56ec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:41.222307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee3972-2807-49b8-8277-006740c56ec4 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:41.224650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0da2d-934f-4735-9824-0f0b40a4eb8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:41.225296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0da2d-934f-4735-9824-0f0b40a4eb8c service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:41.225580 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0da2d-934f-4735-9824-0f0b40a4eb8c service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:41.225857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0da2d-934f-4735-9824-0f0b40a4eb8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:41.228327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-28d0da2d-934f-4735-9824-0f0b40a4eb8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:41.247027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1192/2390] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 13:59:41 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:59:41.247960 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee3972-2807-49b8-8277-006740c56ec4 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:41.257687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee3972-2807-49b8-8277-006740c56ec4 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:41.262877 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2bbc5-b497-4e91-927c-5b867c62da94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:41.263539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2bbc5-b497-4e91-927c-5b867c62da94 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:41.263841 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2bbc5-b497-4e91-927c-5b867c62da94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:41.264119 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2bbc5-b497-4e91-927c-5b867c62da94 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:41.264869 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2bbc5-b497-4e91-927c-5b867c62da94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:41.266436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2bbc5-b497-4e91-927c-5b867c62da94 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:41.276429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1193/2391] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:59:41.280260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2bbc5-b497-4e91-927c-5b867c62da94 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:41.288729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d0da2d-934f-4735-9824-0f0b40a4eb8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:41.290260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2bbc5-b497-4e91-927c-5b867c62da94 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:41.290990 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d0da2d-934f-4735-9824-0f0b40a4eb8c service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:41.294812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be3a1b-bb93-40fa-91df-819fb86fad0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:41.295481 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be3a1b-bb93-40fa-91df-819fb86fad0d admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:41.295765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be3a1b-bb93-40fa-91df-819fb86fad0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:41.296045 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be3a1b-bb93-40fa-91df-819fb86fad0d admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:41.296754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be3a1b-bb93-40fa-91df-819fb86fad0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:41.298294 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be3a1b-bb93-40fa-91df-819fb86fad0d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:41.300453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1199/2392] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:59:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:59:41.309321 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1194/2393] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 13:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:59:41.313575 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0da2d-934f-4735-9824-0f0b40a4eb8c service nova] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:41.324103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0da2d-934f-4735-9824-0f0b40a4eb8c service nova] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:41.329196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea800fd-7c39-486a-b3d9-4fc07865e55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:41.330225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea800fd-7c39-486a-b3d9-4fc07865e55a admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:41.330632 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea800fd-7c39-486a-b3d9-4fc07865e55a admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:41.331007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea800fd-7c39-486a-b3d9-4fc07865e55a admin admin] PATH_INFO: `/v3/projects/e14dac6dd56c4f93b65ad91673debf71/users/ab1b34a915464792b5d3b74956bc49a2/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:41.346685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea800fd-7c39-486a-b3d9-4fc07865e55a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e14dac6dd56c4f93b65ad91673debf71, user_id=ab1b34a915464792b5d3b74956bc49a2, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:41.349486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea800fd-7c39-486a-b3d9-4fc07865e55a admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:41.366801 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1200/2394] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 13:59:41 2022] PUT /identity/v3/projects/e14dac6dd56c4f93b65ad91673debf71/users/ab1b34a915464792b5d3b74956bc49a2/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:59:41.372844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ac00a-4c4b-4378-b188-5025a4cd87b7 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:41.373091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ac00a-4c4b-4378-b188-5025a4cd87b7 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:41.373395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ac00a-4c4b-4378-b188-5025a4cd87b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:41.393019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-ac8ac00a-4c4b-4378-b188-5025a4cd87b7 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:59:41.394235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ac8ac00a-4c4b-4378-b188-5025a4cd87b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:41.467639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1195/2395] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:59:41.478647 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea800fd-7c39-486a-b3d9-4fc07865e55a admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:41.480680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-eea800fd-7c39-486a-b3d9-4fc07865e55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:41.556214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea800fd-7c39-486a-b3d9-4fc07865e55a admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:41.560515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7233e0-62ae-47dc-84c2-b7cdc84fb674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:41.561209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7233e0-62ae-47dc-84c2-b7cdc84fb674 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:41.561571 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7233e0-62ae-47dc-84c2-b7cdc84fb674 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:41.562062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7233e0-62ae-47dc-84c2-b7cdc84fb674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:41.572935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7233e0-62ae-47dc-84c2-b7cdc84fb674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:41.574574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7233e0-62ae-47dc-84c2-b7cdc84fb674 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:41.585913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1201/2396] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:59:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:59:46.067608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1f553-0015-4235-a4dd-bc5396a7c067 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:46.068538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1f553-0015-4235-a4dd-bc5396a7c067 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:46.069141 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1f553-0015-4235-a4dd-bc5396a7c067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:46.102788 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-5ff1f553-0015-4235-a4dd-bc5396a7c067 None None] MFA Rules not processed for user `ab1b34a915464792b5d3b74956bc49a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:59:46.104862 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5ff1f553-0015-4235-a4dd-bc5396a7c067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:46.178043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:59:46.233521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1196/2397] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:59:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:59:46.249577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7233e0-62ae-47dc-84c2-b7cdc84fb674 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:46.280878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:59:46.329249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7233e0-62ae-47dc-84c2-b7cdc84fb674 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:46.335082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf61fa-8941-4e6d-a2dc-2248b797e33c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:46.335748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf61fa-8941-4e6d-a2dc-2248b797e33c service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:46.336058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf61fa-8941-4e6d-a2dc-2248b797e33c service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:46.336453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf61fa-8941-4e6d-a2dc-2248b797e33c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:46.345432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf61fa-8941-4e6d-a2dc-2248b797e33c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:46.347116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf61fa-8941-4e6d-a2dc-2248b797e33c service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:46.357217 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1202/2398] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:59:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:59:46.480917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf69a0-e7da-45a1-aa47-0537f52a0091 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:46.481281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf69a0-e7da-45a1-aa47-0537f52a0091 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:46.481623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf69a0-e7da-45a1-aa47-0537f52a0091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:46.503433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3acf69a0-e7da-45a1-aa47-0537f52a0091 None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:59:46.504649 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3acf69a0-e7da-45a1-aa47-0537f52a0091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:46.594544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1197/2399] 10.222.0.52 () {58 vars in 967 bytes} [Thu May 19 13:59:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:59:46.671675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf61fa-8941-4e6d-a2dc-2248b797e33c service nova] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:46.673525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9fcf61fa-8941-4e6d-a2dc-2248b797e33c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:46.803764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf61fa-8941-4e6d-a2dc-2248b797e33c service nova] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:46.810523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b845b4-17e9-454a-801a-5f39228c20af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:46.811255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b845b4-17e9-454a-801a-5f39228c20af service swift] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:46.811541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b845b4-17e9-454a-801a-5f39228c20af service swift] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:46.811821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b845b4-17e9-454a-801a-5f39228c20af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:46.821356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b845b4-17e9-454a-801a-5f39228c20af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:46.823132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b845b4-17e9-454a-801a-5f39228c20af service swift] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:46.832201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1203/2400] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 13:59:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:59:47.966913 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315adf0-f2a5-4dbf-88c7-ffdf16e0a29d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:47.967728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315adf0-f2a5-4dbf-88c7-ffdf16e0a29d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:47.968173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315adf0-f2a5-4dbf-88c7-ffdf16e0a29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:47.991690 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3315adf0-f2a5-4dbf-88c7-ffdf16e0a29d None None] MFA Rules not processed for user `ab1b34a915464792b5d3b74956bc49a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:59:47.993554 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3315adf0-f2a5-4dbf-88c7-ffdf16e0a29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:48.001512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1198/2401] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:59:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:59:48.781071 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b845b4-17e9-454a-801a-5f39228c20af service swift] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:48.791783 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b845b4-17e9-454a-801a-5f39228c20af service swift] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:48.795910 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f82ae2-b621-40b0-af49-1231fef6987b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:48.796948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f82ae2-b621-40b0-af49-1231fef6987b service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:48.797330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f82ae2-b621-40b0-af49-1231fef6987b service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:48.797708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f82ae2-b621-40b0-af49-1231fef6987b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:48.800161 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-35f82ae2-b621-40b0-af49-1231fef6987b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:48.812266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f82ae2-b621-40b0-af49-1231fef6987b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:48.813835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f82ae2-b621-40b0-af49-1231fef6987b service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:48.824191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1204/2402] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:59:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:59:49.440459 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315adf0-f2a5-4dbf-88c7-ffdf16e0a29d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:49.442064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3315adf0-f2a5-4dbf-88c7-ffdf16e0a29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:49.585986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315adf0-f2a5-4dbf-88c7-ffdf16e0a29d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:49.599162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017e18f-25e3-41e7-8242-911ac5270754 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b105f8163ce84a9ba725e6b04d348446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:49.599848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e18f-25e3-41e7-8242-911ac5270754 service placement] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:49.600140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e18f-25e3-41e7-8242-911ac5270754 service placement] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:49.600433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e18f-25e3-41e7-8242-911ac5270754 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:49.603550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-4017e18f-25e3-41e7-8242-911ac5270754 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:49.618206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017e18f-25e3-41e7-8242-911ac5270754 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:49.620658 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017e18f-25e3-41e7-8242-911ac5270754 service placement] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:49.636975 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1199/2403] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:59:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:59:51.129201 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f82ae2-b621-40b0-af49-1231fef6987b service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:51.131463 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-35f82ae2-b621-40b0-af49-1231fef6987b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:51.170785 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f82ae2-b621-40b0-af49-1231fef6987b service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:51.185133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2af048-bb37-4d23-8cd6-43019d006739 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:51.187335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af048-bb37-4d23-8cd6-43019d006739 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:51.188149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af048-bb37-4d23-8cd6-43019d006739 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:51.188149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af048-bb37-4d23-8cd6-43019d006739 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:51.194077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8c2af048-bb37-4d23-8cd6-43019d006739 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:51.229611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2af048-bb37-4d23-8cd6-43019d006739 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:51.231598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2af048-bb37-4d23-8cd6-43019d006739 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:51.266703 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1205/2404] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:59:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:59:54.042344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017e18f-25e3-41e7-8242-911ac5270754 service placement] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:54.074896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017e18f-25e3-41e7-8242-911ac5270754 service placement] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:54.088153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd4dde-5ad7-4a7f-95b0-a1f5a997d550 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:54.089204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd4dde-5ad7-4a7f-95b0-a1f5a997d550 service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:54.089641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd4dde-5ad7-4a7f-95b0-a1f5a997d550 service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:54.090080 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd4dde-5ad7-4a7f-95b0-a1f5a997d550 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:54.093230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c3dd4dde-5ad7-4a7f-95b0-a1f5a997d550 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:54.115449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd4dde-5ad7-4a7f-95b0-a1f5a997d550 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:54.117344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd4dde-5ad7-4a7f-95b0-a1f5a997d550 service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:54.131437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1200/2405] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 13:59:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:59:55.725142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964dba3-44a9-4365-83f3-e5434ec5128c None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:55.725577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964dba3-44a9-4365-83f3-e5434ec5128c None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:55.725839 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964dba3-44a9-4365-83f3-e5434ec5128c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:55.751352 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-4964dba3-44a9-4365-83f3-e5434ec5128c None None] MFA Rules not processed for user `0cc3e0105f2a472092423852b1f39979`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:59:55.752590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4964dba3-44a9-4365-83f3-e5434ec5128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:55.823157 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1206/2406] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 13:59:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:59:58.068886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd4dde-5ad7-4a7f-95b0-a1f5a997d550 service nova] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:58.071287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c3dd4dde-5ad7-4a7f-95b0-a1f5a997d550 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:58.090507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd4dde-5ad7-4a7f-95b0-a1f5a997d550 service nova] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:58.095802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:58.096672 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:58.097142 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:58.097467 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] PATH_INFO: `/v3/users/a1274435c6774faa9eeeac6726b52b47` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:58.098892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1274435c6774faa9eeeac6726b52b47)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:58.101278 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:58.145884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:59:58.154990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1274435c6774faa9eeeac6726b52b47'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.164287 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1274435c6774faa9eeeac6726b52b47'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.165075 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1274435c6774faa9eeeac6726b52b47'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.173758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1274435c6774faa9eeeac6726b52b47'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.192248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1274435c6774faa9eeeac6726b52b47'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.199750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1201/2407] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:59:58 2022] DELETE /identity/v3/users/a1274435c6774faa9eeeac6726b52b47 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:59:58.459840 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964dba3-44a9-4365-83f3-e5434ec5128c None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:58.477109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964dba3-44a9-4365-83f3-e5434ec5128c None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:58.494817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:59:58.558697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:58.559960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:58.560877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:58.561362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] PATH_INFO: `/v3/projects/a51cbae6cb494a2fac656e416fe868c2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:58.563415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] RBAC: Authorizing `identity:delete_project(project_id=a51cbae6cb494a2fac656e416fe868c2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:58.565633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:58.636519 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a51cbae6cb494a2fac656e416fe868c2', 'request_id': 'req-659163f4-ea18-4b26-a5ea-e2826f4fecec'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.644875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a51cbae6cb494a2fac656e416fe868c2', 'request_id': 'req-659163f4-ea18-4b26-a5ea-e2826f4fecec'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.657059 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a51cbae6cb494a2fac656e416fe868c2', 'request_id': 'req-659163f4-ea18-4b26-a5ea-e2826f4fecec'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.673768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] The token cache is being invalidate because project a51cbae6cb494a2fac656e416fe868c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:59:58.674223 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.675290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.680755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1207/2408] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:59:58 2022] DELETE /identity/v3/projects/a51cbae6cb494a2fac656e416fe868c2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:59:58.682000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:58.683512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:58.704533 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33471e-8c1b-41db-bb0c-7756842fcad3 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:58.796470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:58.797417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:58.797760 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:58.798069 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] PATH_INFO: `/v3/users/c04a766278dd451da58e003d989f9959` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:58.800081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c04a766278dd451da58e003d989f9959)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:58.802574 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:58.872632 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c04a766278dd451da58e003d989f9959'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.882663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c04a766278dd451da58e003d989f9959'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.884549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c04a766278dd451da58e003d989f9959'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.898011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c04a766278dd451da58e003d989f9959'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.912942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c04a766278dd451da58e003d989f9959'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:58.920992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1202/2409] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 13:59:58 2022] DELETE /identity/v3/users/c04a766278dd451da58e003d989f9959 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:59:59.174011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:59:59.175536 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:59:59.207623 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659163f4-ea18-4b26-a5ea-e2826f4fecec admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:59:59.297081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:59:59.297963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:59:59.298247 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:59:59.298522 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] PATH_INFO: `/v3/projects/1f4275b742b84ecc97c6aff544e82d88` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:59:59.299830 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4275b742b84ecc97c6aff544e82d88)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:59:59.301772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:59:59.356708 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4275b742b84ecc97c6aff544e82d88', 'request_id': 'req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:59.359321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:59:59.363526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4275b742b84ecc97c6aff544e82d88', 'request_id': 'req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:59.375262 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4275b742b84ecc97c6aff544e82d88', 'request_id': 'req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:59.406277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] The token cache is being invalidate because project 1f4275b742b84ecc97c6aff544e82d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:59:59.407226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:59.407983 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:59:59.414412 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1208/2410] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 13:59:59 2022] DELETE /identity/v3/projects/1f4275b742b84ecc97c6aff544e82d88 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:00:04.865983 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:00:04.867510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:00:04.947700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:00:04.999706 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7f04b-9300-4c62-8bdb-0378e7d0c7ad admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:00:05.072556 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2659f-243f-45c2-a061-f883497ffdb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:00:05.073226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2659f-243f-45c2-a061-f883497ffdb2 service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:05.073664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2659f-243f-45c2-a061-f883497ffdb2 service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:05.073950 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2659f-243f-45c2-a061-f883497ffdb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:05.076324 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5aa2659f-243f-45c2-a061-f883497ffdb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:00:05.089670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2659f-243f-45c2-a061-f883497ffdb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:00:05.091234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2659f-243f-45c2-a061-f883497ffdb2 service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:00:05.103448 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1203/2411] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:00:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:00:05.956072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:00:05.957418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:00:05.973311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccabea-cc25-4461-a5cc-f67b6d53fdb8 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:00:05.979731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1c7d8-25d2-4aea-b1a7-5be3b12dd710 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:00:05.980380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1c7d8-25d2-4aea-b1a7-5be3b12dd710 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:05.980665 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1c7d8-25d2-4aea-b1a7-5be3b12dd710 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:05.980962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1c7d8-25d2-4aea-b1a7-5be3b12dd710 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:05.982673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-13a1c7d8-25d2-4aea-b1a7-5be3b12dd710 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:00:06.045533 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1c7d8-25d2-4aea-b1a7-5be3b12dd710 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:00:06.047149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1c7d8-25d2-4aea-b1a7-5be3b12dd710 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:00:06.057386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1209/2412] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:00:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:00:15.098781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=46 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:00:16.052148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:DELETE=3,keystone:SELECT=34,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:00:21.186258 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a01935-cda2-4957-bb82-59265c48657a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:21.186718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a01935-cda2-4957-bb82-59265c48657a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:21.186811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a01935-cda2-4957-bb82-59265c48657a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:21.204906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-f2a01935-cda2-4957-bb82-59265c48657a None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:00:21.206294 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f2a01935-cda2-4957-bb82-59265c48657a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:00:21.266514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1204/2413] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 14:00:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:00:21.279389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1c7d8-25d2-4aea-b1a7-5be3b12dd710 service nova] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:00:21.280948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-13a1c7d8-25d2-4aea-b1a7-5be3b12dd710 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:00:21.350977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1c7d8-25d2-4aea-b1a7-5be3b12dd710 service nova] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:00:21.356422 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a518cc7-848f-481b-9f57-0f00119596c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:00:21.357088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a518cc7-848f-481b-9f57-0f00119596c8 service swift] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:21.357372 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a518cc7-848f-481b-9f57-0f00119596c8 service swift] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:21.357649 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a518cc7-848f-481b-9f57-0f00119596c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:21.365627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a518cc7-848f-481b-9f57-0f00119596c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:00:21.367225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a518cc7-848f-481b-9f57-0f00119596c8 service swift] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:00:21.375842 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1210/2414] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 14:00:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:00:31.259434 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=25 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:00:31.372634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=32 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:00:37.036086 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fe0e6-1059-4fb0-bad5-2ff93f9f34e6 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:37.036570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fe0e6-1059-4fb0-bad5-2ff93f9f34e6 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:37.036656 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fe0e6-1059-4fb0-bad5-2ff93f9f34e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:37.060369 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-733fe0e6-1059-4fb0-bad5-2ff93f9f34e6 None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:00:37.061575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-733fe0e6-1059-4fb0-bad5-2ff93f9f34e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:00:37.068757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1205/2415] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 14:00:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:00:37.080873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a518cc7-848f-481b-9f57-0f00119596c8 service swift] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:00:37.091823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a518cc7-848f-481b-9f57-0f00119596c8 service swift] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:00:37.096776 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace715e-2f85-4704-85ac-e05c1419b4c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:00:37.097405 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace715e-2f85-4704-85ac-e05c1419b4c5 service swift] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:37.097686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace715e-2f85-4704-85ac-e05c1419b4c5 service swift] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:37.097958 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace715e-2f85-4704-85ac-e05c1419b4c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:37.106646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace715e-2f85-4704-85ac-e05c1419b4c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:00:37.108253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace715e-2f85-4704-85ac-e05c1419b4c5 service swift] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:00:37.116185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1211/2416] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 14:00:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:00:47.051361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:00:47.112951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=6 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:00:51.689025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03def6b4-6233-43e1-95ab-829a5dbcba0b None None] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:51.689805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03def6b4-6233-43e1-95ab-829a5dbcba0b None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:51.690170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03def6b4-6233-43e1-95ab-829a5dbcba0b None None] PATH_INFO: `/` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:51.691778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1206/2417] 10.222.0.52 () {54 vars in 921 bytes} [Thu May 19 14:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:00:51.698524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f5eb7-a721-4747-8da7-a804649fcb4f None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:51.699419 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f5eb7-a721-4747-8da7-a804649fcb4f None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:51.700002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f5eb7-a721-4747-8da7-a804649fcb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:51.718984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-4e8f5eb7-a721-4747-8da7-a804649fcb4f None None] MFA Rules not processed for user `f18f635bff0142409e90da66547927f1`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:00:51.720212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4e8f5eb7-a721-4747-8da7-a804649fcb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:00:51.780349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1212/2418] 10.222.0.52 () {58 vars in 1018 bytes} [Thu May 19 14:00:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:00:51.792357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03def6b4-6233-43e1-95ab-829a5dbcba0b None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:00:51.794064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-03def6b4-6233-43e1-95ab-829a5dbcba0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:00:51.807519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03def6b4-6233-43e1-95ab-829a5dbcba0b None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:00:51.811041 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376cbba-f0c9-43ea-ad04-42e1cc1878ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:00:51.811683 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376cbba-f0c9-43ea-ad04-42e1cc1878ac service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:51.811962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376cbba-f0c9-43ea-ad04-42e1cc1878ac service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:51.812232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376cbba-f0c9-43ea-ad04-42e1cc1878ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:51.819933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376cbba-f0c9-43ea-ad04-42e1cc1878ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:00:51.821920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376cbba-f0c9-43ea-ad04-42e1cc1878ac service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:00:51.830021 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1207/2419] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:00:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:00:51.917703 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff297e-371d-4770-8b85-6c71e59dbef1 None None] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:51.917999 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff297e-371d-4770-8b85-6c71e59dbef1 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:51.918285 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff297e-371d-4770-8b85-6c71e59dbef1 None None] PATH_INFO: `/` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:51.919707 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1213/2420] 10.222.0.52 () {54 vars in 921 bytes} [Thu May 19 14:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 14:00:51.925173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aad3d0-3423-4ab9-83d1-92163b4fea92 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:51.925455 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aad3d0-3423-4ab9-83d1-92163b4fea92 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:51.925730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aad3d0-3423-4ab9-83d1-92163b4fea92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:51.944098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a0aad3d0-3423-4ab9-83d1-92163b4fea92 None None] MFA Rules not processed for user `4f6093d0c1fc41d98c58bace81e3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:00:51.945282 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a0aad3d0-3423-4ab9-83d1-92163b4fea92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:00:51.955413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1208/2421] 10.222.0.52 () {58 vars in 1018 bytes} [Thu May 19 14:00:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:00:51.965971 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff297e-371d-4770-8b85-6c71e59dbef1 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:00:51.967735 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-83ff297e-371d-4770-8b85-6c71e59dbef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:00:51.981608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff297e-371d-4770-8b85-6c71e59dbef1 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:00:51.985277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bbd37-ad73-43c6-974e-27bd98ecc5ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:00:51.985907 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bbd37-ad73-43c6-974e-27bd98ecc5ef service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:51.986192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bbd37-ad73-43c6-974e-27bd98ecc5ef service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:51.986470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bbd37-ad73-43c6-974e-27bd98ecc5ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:51.997388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bbd37-ad73-43c6-974e-27bd98ecc5ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:00:51.998811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bbd37-ad73-43c6-974e-27bd98ecc5ef service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:00:52.010379 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1214/2422] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:00:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:00:54.504030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d068646-5474-4e5b-abb1-bb17b507ef0c None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:54.504505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d068646-5474-4e5b-abb1-bb17b507ef0c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:54.504599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d068646-5474-4e5b-abb1-bb17b507ef0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:54.522914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-2d068646-5474-4e5b-abb1-bb17b507ef0c None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:00:54.524183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2d068646-5474-4e5b-abb1-bb17b507ef0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:00:54.531563 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1209/2423] 10.222.0.52 () {58 vars in 967 bytes} [Thu May 19 14:00:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:00:54.552955 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bbd37-ad73-43c6-974e-27bd98ecc5ef service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:00:54.564752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bbd37-ad73-43c6-974e-27bd98ecc5ef service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:00:54.569688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f108e13-3189-4b1c-907c-e67005198011 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:00:54.570317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f108e13-3189-4b1c-907c-e67005198011 service swift] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:00:54.570603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f108e13-3189-4b1c-907c-e67005198011 service swift] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:00:54.570932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f108e13-3189-4b1c-907c-e67005198011 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:00:54.578929 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f108e13-3189-4b1c-907c-e67005198011 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:00:54.580504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f108e13-3189-4b1c-907c-e67005198011 service swift] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:00:54.589075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1215/2424] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 14:00:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:01:04.517758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=12 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:04.585295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=35 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:09.670945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d068646-5474-4e5b-abb1-bb17b507ef0c None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:09.680212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d068646-5474-4e5b-abb1-bb17b507ef0c None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:09.684478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b32f78-a4c1-4c49-bfb5-c89b1936d49c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:09.685116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b32f78-a4c1-4c49-bfb5-c89b1936d49c service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:09.685397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b32f78-a4c1-4c49-bfb5-c89b1936d49c service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:09.685673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b32f78-a4c1-4c49-bfb5-c89b1936d49c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:09.687443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-92b32f78-a4c1-4c49-bfb5-c89b1936d49c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:09.748175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b32f78-a4c1-4c49-bfb5-c89b1936d49c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:09.749595 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b32f78-a4c1-4c49-bfb5-c89b1936d49c service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:09.759207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1210/2425] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:13.070438 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f108e13-3189-4b1c-907c-e67005198011 service swift] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:13.074494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-6f108e13-3189-4b1c-907c-e67005198011 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:13.092908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f108e13-3189-4b1c-907c-e67005198011 service swift] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:13.100506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a662290-e1bf-47ef-8da7-bd8e1819735c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:13.101584 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a662290-e1bf-47ef-8da7-bd8e1819735c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:13.102018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a662290-e1bf-47ef-8da7-bd8e1819735c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:13.102441 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a662290-e1bf-47ef-8da7-bd8e1819735c admin admin] PATH_INFO: `/v3/users/0cc3e0105f2a472092423852b1f39979` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:13.104515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a662290-e1bf-47ef-8da7-bd8e1819735c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cc3e0105f2a472092423852b1f39979)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:13.106602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a662290-e1bf-47ef-8da7-bd8e1819735c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:13.176551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4a662290-e1bf-47ef-8da7-bd8e1819735c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cc3e0105f2a472092423852b1f39979'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:13.186443 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4a662290-e1bf-47ef-8da7-bd8e1819735c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cc3e0105f2a472092423852b1f39979'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:13.195984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4a662290-e1bf-47ef-8da7-bd8e1819735c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cc3e0105f2a472092423852b1f39979'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:13.196750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4a662290-e1bf-47ef-8da7-bd8e1819735c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cc3e0105f2a472092423852b1f39979'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:13.198061 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4a662290-e1bf-47ef-8da7-bd8e1819735c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cc3e0105f2a472092423852b1f39979'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:13.217168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1216/2426] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:13 2022] DELETE /identity/v3/users/0cc3e0105f2a472092423852b1f39979 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:13.393256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b32f78-a4c1-4c49-bfb5-c89b1936d49c service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:13.404669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b32f78-a4c1-4c49-bfb5-c89b1936d49c service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:13.462169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a540609-18da-4728-91ef-b5e4415bfb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:13.463648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a540609-18da-4728-91ef-b5e4415bfb46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:13.464116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a540609-18da-4728-91ef-b5e4415bfb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:13.464436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a540609-18da-4728-91ef-b5e4415bfb46 admin admin] PATH_INFO: `/v3/projects/89b4336043d5446b922432a2f43af17a` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:13.465836 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a540609-18da-4728-91ef-b5e4415bfb46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89b4336043d5446b922432a2f43af17a)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:13.467764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a540609-18da-4728-91ef-b5e4415bfb46 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:13.513321 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0a540609-18da-4728-91ef-b5e4415bfb46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89b4336043d5446b922432a2f43af17a', 'request_id': 'req-0a540609-18da-4728-91ef-b5e4415bfb46'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:13.519383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0a540609-18da-4728-91ef-b5e4415bfb46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89b4336043d5446b922432a2f43af17a', 'request_id': 'req-0a540609-18da-4728-91ef-b5e4415bfb46'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:13.527412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0a540609-18da-4728-91ef-b5e4415bfb46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89b4336043d5446b922432a2f43af17a', 'request_id': 'req-0a540609-18da-4728-91ef-b5e4415bfb46'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:13.537819 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0a540609-18da-4728-91ef-b5e4415bfb46 admin admin] The token cache is being invalidate because project 89b4336043d5446b922432a2f43af17a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:13.538109 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0a540609-18da-4728-91ef-b5e4415bfb46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:13.538725 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0a540609-18da-4728-91ef-b5e4415bfb46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:13.540297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1211/2427] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:13 2022] DELETE /identity/v3/projects/89b4336043d5446b922432a2f43af17a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:13.545919 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a0a1e-f548-438a-8f91-3d89e84e98d6 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:13.546214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a0a1e-f548-438a-8f91-3d89e84e98d6 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:13.546498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a0a1e-f548-438a-8f91-3d89e84e98d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:13.564484 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e61a0a1e-f548-438a-8f91-3d89e84e98d6 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:13.565688 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e61a0a1e-f548-438a-8f91-3d89e84e98d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:13.623577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1217/2428] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:13.632088 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0760702-821d-48d5-bddd-4d10f2637902 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:13.632377 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0760702-821d-48d5-bddd-4d10f2637902 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:13.632655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0760702-821d-48d5-bddd-4d10f2637902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:13.652051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-f0760702-821d-48d5-bddd-4d10f2637902 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:13.653243 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f0760702-821d-48d5-bddd-4d10f2637902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:13.661069 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1212/2429] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:13.665172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a0a1e-f548-438a-8f91-3d89e84e98d6 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:13.675636 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a0a1e-f548-438a-8f91-3d89e84e98d6 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:13.680394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313df4f4-7e8c-450b-9b58-e578325cb846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:13.681069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313df4f4-7e8c-450b-9b58-e578325cb846 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:13.681354 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313df4f4-7e8c-450b-9b58-e578325cb846 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:13.681636 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313df4f4-7e8c-450b-9b58-e578325cb846 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:13.682386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313df4f4-7e8c-450b-9b58-e578325cb846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:01:13.682699 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313df4f4-7e8c-450b-9b58-e578325cb846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:13.684328 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313df4f4-7e8c-450b-9b58-e578325cb846 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:13.695836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1218/2430] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:01:13.700583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0760702-821d-48d5-bddd-4d10f2637902 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:13.709528 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0760702-821d-48d5-bddd-4d10f2637902 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:13.714782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475da84f-8963-43bb-a36d-0a45aaaba88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:13.715448 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475da84f-8963-43bb-a36d-0a45aaaba88c admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:13.715741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475da84f-8963-43bb-a36d-0a45aaaba88c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:13.716004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475da84f-8963-43bb-a36d-0a45aaaba88c admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:13.716687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475da84f-8963-43bb-a36d-0a45aaaba88c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:13.718570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475da84f-8963-43bb-a36d-0a45aaaba88c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:13.732148 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:13.754928 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1213/2431] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:13 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:13.759312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313df4f4-7e8c-450b-9b58-e578325cb846 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:13.770469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313df4f4-7e8c-450b-9b58-e578325cb846 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:13.826662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe46216-2e60-4f92-9b6c-962d2d362e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:13.827362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe46216-2e60-4f92-9b6c-962d2d362e97 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:13.827653 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe46216-2e60-4f92-9b6c-962d2d362e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:13.827958 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe46216-2e60-4f92-9b6c-962d2d362e97 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:13.828662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe46216-2e60-4f92-9b6c-962d2d362e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:13.830598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe46216-2e60-4f92-9b6c-962d2d362e97 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:13.837169 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:13.854969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1219/2432] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:13 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:13.859048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475da84f-8963-43bb-a36d-0a45aaaba88c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:13.869548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475da84f-8963-43bb-a36d-0a45aaaba88c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:13.874169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84994240-5560-4531-a69c-8c4f6ff9980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:13.874816 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84994240-5560-4531-a69c-8c4f6ff9980a admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:13.875132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84994240-5560-4531-a69c-8c4f6ff9980a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:13.875407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84994240-5560-4531-a69c-8c4f6ff9980a admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:13.876124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84994240-5560-4531-a69c-8c4f6ff9980a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:13.877696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84994240-5560-4531-a69c-8c4f6ff9980a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:13.888267 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1214/2433] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:13.892123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe46216-2e60-4f92-9b6c-962d2d362e97 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:13.903052 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe46216-2e60-4f92-9b6c-962d2d362e97 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:13.907798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f734af-3451-4b79-a728-967bce866aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:13.908452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f734af-3451-4b79-a728-967bce866aef admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:13.908737 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f734af-3451-4b79-a728-967bce866aef admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:13.909016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f734af-3451-4b79-a728-967bce866aef admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:13.909774 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f734af-3451-4b79-a728-967bce866aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:13.911409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f734af-3451-4b79-a728-967bce866aef admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:13.922085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1220/2434] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:13.926135 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84994240-5560-4531-a69c-8c4f6ff9980a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:13.936261 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84994240-5560-4531-a69c-8c4f6ff9980a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:13.940826 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4cd42-c313-4469-b072-55d3964b3617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:13.941685 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4cd42-c313-4469-b072-55d3964b3617 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:13.941967 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4cd42-c313-4469-b072-55d3964b3617 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:13.942240 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4cd42-c313-4469-b072-55d3964b3617 admin admin] PATH_INFO: `/v3/projects/62a4a4e85439443f954373849679899a/users/cdb8b4f13f4f4ca58230d762a110a2b6/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:13.957271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4cd42-c313-4469-b072-55d3964b3617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a4a4e85439443f954373849679899a, user_id=cdb8b4f13f4f4ca58230d762a110a2b6, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:13.959936 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4cd42-c313-4469-b072-55d3964b3617 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:13.973958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1215/2435] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:13 2022] PUT /identity/v3/projects/62a4a4e85439443f954373849679899a/users/cdb8b4f13f4f4ca58230d762a110a2b6/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:01:13.984033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9bcff-126e-4522-97cf-9428665d5d6e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:13.984327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9bcff-126e-4522-97cf-9428665d5d6e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:13.984613 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9bcff-126e-4522-97cf-9428665d5d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:14.014245 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-b2e9bcff-126e-4522-97cf-9428665d5d6e None None] MFA Rules not processed for user `cdb8b4f13f4f4ca58230d762a110a2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:14.015547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-b2e9bcff-126e-4522-97cf-9428665d5d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:14.086606 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1221/2436] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:14.091939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4cd42-c313-4469-b072-55d3964b3617 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:14.170084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4cd42-c313-4469-b072-55d3964b3617 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:14.174976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30c835-0268-4bf7-97f1-517bf2243d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:14.175667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30c835-0268-4bf7-97f1-517bf2243d71 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:14.175953 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30c835-0268-4bf7-97f1-517bf2243d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:14.176234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30c835-0268-4bf7-97f1-517bf2243d71 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:14.177067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30c835-0268-4bf7-97f1-517bf2243d71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:14.179025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30c835-0268-4bf7-97f1-517bf2243d71 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:14.204333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1216/2437] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:14 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:14.208305 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9bcff-126e-4522-97cf-9428665d5d6e None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:14.219579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9bcff-126e-4522-97cf-9428665d5d6e None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:14.276852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d385b6-0b41-4c36-85c2-65078659a3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:14.277551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d385b6-0b41-4c36-85c2-65078659a3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:14.277842 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d385b6-0b41-4c36-85c2-65078659a3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:14.278136 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d385b6-0b41-4c36-85c2-65078659a3a5 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:14.278815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d385b6-0b41-4c36-85c2-65078659a3a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:14.280816 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d385b6-0b41-4c36-85c2-65078659a3a5 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:14.300742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1222/2438] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:14 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:14.304670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30c835-0268-4bf7-97f1-517bf2243d71 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:14.315103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30c835-0268-4bf7-97f1-517bf2243d71 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:14.319726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd80d03-e601-4c16-8925-0e996f825809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:14.320379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd80d03-e601-4c16-8925-0e996f825809 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:14.320663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd80d03-e601-4c16-8925-0e996f825809 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:14.320940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd80d03-e601-4c16-8925-0e996f825809 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:14.321655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd80d03-e601-4c16-8925-0e996f825809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:14.323581 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd80d03-e601-4c16-8925-0e996f825809 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:14.333903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1217/2439] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:14.338598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d385b6-0b41-4c36-85c2-65078659a3a5 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:14.348651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d385b6-0b41-4c36-85c2-65078659a3a5 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:14.353845 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a9c93-8afd-47e8-bf51-27b3a1853485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:14.354713 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a9c93-8afd-47e8-bf51-27b3a1853485 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:14.355035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a9c93-8afd-47e8-bf51-27b3a1853485 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:14.355316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a9c93-8afd-47e8-bf51-27b3a1853485 admin admin] PATH_INFO: `/v3/projects/ef582cd99eaa4b07b17f91c0c46bf7f8/users/5f1ee7d06ec5475eabd30b6b284aeb02/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:14.369818 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a9c93-8afd-47e8-bf51-27b3a1853485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef582cd99eaa4b07b17f91c0c46bf7f8, user_id=5f1ee7d06ec5475eabd30b6b284aeb02, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:14.372521 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a9c93-8afd-47e8-bf51-27b3a1853485 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:14.388601 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1223/2440] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:14 2022] PUT /identity/v3/projects/ef582cd99eaa4b07b17f91c0c46bf7f8/users/5f1ee7d06ec5475eabd30b6b284aeb02/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:14.392739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd80d03-e601-4c16-8925-0e996f825809 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:14.404319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd80d03-e601-4c16-8925-0e996f825809 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:14.460839 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72f143-9901-4b3e-a74b-79868c41bb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:14.461529 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72f143-9901-4b3e-a74b-79868c41bb4a admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:14.461812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72f143-9901-4b3e-a74b-79868c41bb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:14.462087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72f143-9901-4b3e-a74b-79868c41bb4a admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:14.463389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72f143-9901-4b3e-a74b-79868c41bb4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:14.464966 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72f143-9901-4b3e-a74b-79868c41bb4a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:14.474474 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1218/2441] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:14.478692 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a9c93-8afd-47e8-bf51-27b3a1853485 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:14.489194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a9c93-8afd-47e8-bf51-27b3a1853485 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:14.493882 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972ef5d-f7f7-4f0a-bf9f-fee0ca604d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:14.494732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972ef5d-f7f7-4f0a-bf9f-fee0ca604d21 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:14.495047 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972ef5d-f7f7-4f0a-bf9f-fee0ca604d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:14.495327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972ef5d-f7f7-4f0a-bf9f-fee0ca604d21 admin admin] PATH_INFO: `/v3/projects/ef582cd99eaa4b07b17f91c0c46bf7f8/users/5f1ee7d06ec5475eabd30b6b284aeb02/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:14.497303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972ef5d-f7f7-4f0a-bf9f-fee0ca604d21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef582cd99eaa4b07b17f91c0c46bf7f8, user_id=5f1ee7d06ec5475eabd30b6b284aeb02, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:14.499959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972ef5d-f7f7-4f0a-bf9f-fee0ca604d21 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:14.510635 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1224/2442] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:14 2022] PUT /identity/v3/projects/ef582cd99eaa4b07b17f91c0c46bf7f8/users/5f1ee7d06ec5475eabd30b6b284aeb02/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:14.521807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c31a15-620d-4169-92e6-7ad821ac5bb4 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:14.522106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c31a15-620d-4169-92e6-7ad821ac5bb4 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:14.522395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c31a15-620d-4169-92e6-7ad821ac5bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:14.549753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-d6c31a15-620d-4169-92e6-7ad821ac5bb4 None None] MFA Rules not processed for user `5f1ee7d06ec5475eabd30b6b284aeb02`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:14.551049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d6c31a15-620d-4169-92e6-7ad821ac5bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:14.560147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:14.619799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1219/2443] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:14.631004 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972ef5d-f7f7-4f0a-bf9f-fee0ca604d21 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:14.632976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4972ef5d-f7f7-4f0a-bf9f-fee0ca604d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:14.658046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:14.694197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972ef5d-f7f7-4f0a-bf9f-fee0ca604d21 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:14.698386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09716bd-5661-433f-8c0b-e10343aea1ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:14.699070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09716bd-5661-433f-8c0b-e10343aea1ec service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:14.699356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09716bd-5661-433f-8c0b-e10343aea1ec service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:14.699637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09716bd-5661-433f-8c0b-e10343aea1ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:14.708779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09716bd-5661-433f-8c0b-e10343aea1ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:14.710196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09716bd-5661-433f-8c0b-e10343aea1ec service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:14.720317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1225/2444] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:14.786330 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c31a15-620d-4169-92e6-7ad821ac5bb4 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:14.847817 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c31a15-620d-4169-92e6-7ad821ac5bb4 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:14.852301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:14.853160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:14.853520 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:14.853793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] PATH_INFO: `/v3/users/cdb8b4f13f4f4ca58230d762a110a2b6` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:14.855009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb8b4f13f4f4ca58230d762a110a2b6)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:14.856905 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:14.903794 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb8b4f13f4f4ca58230d762a110a2b6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:14.910618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb8b4f13f4f4ca58230d762a110a2b6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:14.911379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb8b4f13f4f4ca58230d762a110a2b6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:14.918814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb8b4f13f4f4ca58230d762a110a2b6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:14.930471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb8b4f13f4f4ca58230d762a110a2b6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:14.937022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1220/2445] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:14 2022] DELETE /identity/v3/users/cdb8b4f13f4f4ca58230d762a110a2b6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:14.941636 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333edc1-f8dc-4919-9004-7a897b3b561e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:14.941930 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333edc1-f8dc-4919-9004-7a897b3b561e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:14.942209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333edc1-f8dc-4919-9004-7a897b3b561e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:14.959823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-0333edc1-f8dc-4919-9004-7a897b3b561e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:14.961017 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-0333edc1-f8dc-4919-9004-7a897b3b561e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:15.019633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1226/2446] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:15.029717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:15.031158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:15.063319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:15.091675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13939b70-b5c6-479a-975e-722b8b53bf74 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:15.095361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd820c8-598d-443e-a2f9-b516ce5db24c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:15.096004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd820c8-598d-443e-a2f9-b516ce5db24c service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:15.096289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd820c8-598d-443e-a2f9-b516ce5db24c service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:15.096565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd820c8-598d-443e-a2f9-b516ce5db24c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:15.105911 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd820c8-598d-443e-a2f9-b516ce5db24c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:15.107300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd820c8-598d-443e-a2f9-b516ce5db24c service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:15.116692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1221/2447] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:15.421731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333edc1-f8dc-4919-9004-7a897b3b561e None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:15.423421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-0333edc1-f8dc-4919-9004-7a897b3b561e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:15.441373 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333edc1-f8dc-4919-9004-7a897b3b561e None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:15.448159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:15.449370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:15.449917 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:15.450326 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] PATH_INFO: `/v3/projects/62a4a4e85439443f954373849679899a` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:15.451706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a4a4e85439443f954373849679899a)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:15.453773 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:15.505035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a4a4e85439443f954373849679899a', 'request_id': 'req-6172f210-c976-4dc9-9b6f-5eb632f0fb99'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:15.512548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a4a4e85439443f954373849679899a', 'request_id': 'req-6172f210-c976-4dc9-9b6f-5eb632f0fb99'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:15.522000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a4a4e85439443f954373849679899a', 'request_id': 'req-6172f210-c976-4dc9-9b6f-5eb632f0fb99'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:15.535875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] The token cache is being invalidate because project 62a4a4e85439443f954373849679899a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:15.536173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:15.536856 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:15.538701 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1227/2448] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:15 2022] DELETE /identity/v3/projects/62a4a4e85439443f954373849679899a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:15.542716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd820c8-598d-443e-a2f9-b516ce5db24c service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:15.552919 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd820c8-598d-443e-a2f9-b516ce5db24c service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:15.621587 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11551356-004c-4154-971c-c292ebb768b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:15.622461 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11551356-004c-4154-971c-c292ebb768b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:15.622752 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11551356-004c-4154-971c-c292ebb768b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:15.623064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11551356-004c-4154-971c-c292ebb768b7 admin admin] PATH_INFO: `/v3/users/5f1ee7d06ec5475eabd30b6b284aeb02` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:15.624279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11551356-004c-4154-971c-c292ebb768b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f1ee7d06ec5475eabd30b6b284aeb02)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:15.626180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11551356-004c-4154-971c-c292ebb768b7 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:15.682716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-11551356-004c-4154-971c-c292ebb768b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f1ee7d06ec5475eabd30b6b284aeb02'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:15.693311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-11551356-004c-4154-971c-c292ebb768b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f1ee7d06ec5475eabd30b6b284aeb02'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:15.694113 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-11551356-004c-4154-971c-c292ebb768b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f1ee7d06ec5475eabd30b6b284aeb02'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:15.702814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-11551356-004c-4154-971c-c292ebb768b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f1ee7d06ec5475eabd30b6b284aeb02'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:15.715856 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-11551356-004c-4154-971c-c292ebb768b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f1ee7d06ec5475eabd30b6b284aeb02'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:15.721870 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1222/2449] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:15 2022] DELETE /identity/v3/users/5f1ee7d06ec5475eabd30b6b284aeb02 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:16.152510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:16.154046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:16.173019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172f210-c976-4dc9-9b6f-5eb632f0fb99 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:16.221293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:16.236862 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e0bd7-2899-4fbd-a272-6e0729210a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:16.238033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e0bd7-2899-4fbd-a272-6e0729210a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:16.238315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e0bd7-2899-4fbd-a272-6e0729210a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:16.238595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e0bd7-2899-4fbd-a272-6e0729210a0a admin admin] PATH_INFO: `/v3/projects/ef582cd99eaa4b07b17f91c0c46bf7f8` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:16.239834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e0bd7-2899-4fbd-a272-6e0729210a0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef582cd99eaa4b07b17f91c0c46bf7f8)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:16.241730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e0bd7-2899-4fbd-a272-6e0729210a0a admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:16.289351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-b82e0bd7-2899-4fbd-a272-6e0729210a0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef582cd99eaa4b07b17f91c0c46bf7f8', 'request_id': 'req-b82e0bd7-2899-4fbd-a272-6e0729210a0a'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:16.295022 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-b82e0bd7-2899-4fbd-a272-6e0729210a0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef582cd99eaa4b07b17f91c0c46bf7f8', 'request_id': 'req-b82e0bd7-2899-4fbd-a272-6e0729210a0a'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:16.302521 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-b82e0bd7-2899-4fbd-a272-6e0729210a0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef582cd99eaa4b07b17f91c0c46bf7f8', 'request_id': 'req-b82e0bd7-2899-4fbd-a272-6e0729210a0a'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:16.314214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-b82e0bd7-2899-4fbd-a272-6e0729210a0a admin admin] The token cache is being invalidate because project ef582cd99eaa4b07b17f91c0c46bf7f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:16.314498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-b82e0bd7-2899-4fbd-a272-6e0729210a0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:16.315154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-b82e0bd7-2899-4fbd-a272-6e0729210a0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:16.316676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1228/2450] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:16 2022] DELETE /identity/v3/projects/ef582cd99eaa4b07b17f91c0c46bf7f8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:16.323161 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b7610-aa25-409f-a84c-1337d127a1a2 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:16.323483 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b7610-aa25-409f-a84c-1337d127a1a2 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:16.323784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b7610-aa25-409f-a84c-1337d127a1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:16.343463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-563b7610-aa25-409f-a84c-1337d127a1a2 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:16.344681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-563b7610-aa25-409f-a84c-1337d127a1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:16.389617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:16.406098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1223/2451] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:16.413757 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd1b7e-7f04-4a87-8069-024f5a99a305 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:16.414050 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd1b7e-7f04-4a87-8069-024f5a99a305 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:16.414334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd1b7e-7f04-4a87-8069-024f5a99a305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:16.433014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-8bcd1b7e-7f04-4a87-8069-024f5a99a305 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:16.434214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8bcd1b7e-7f04-4a87-8069-024f5a99a305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:16.442152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1229/2452] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:16.446337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b7610-aa25-409f-a84c-1337d127a1a2 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:16.456758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b7610-aa25-409f-a84c-1337d127a1a2 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:16.461271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c88ee7-1b9a-4df9-8d8b-a8a79b768343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:16.461923 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c88ee7-1b9a-4df9-8d8b-a8a79b768343 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:16.462209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c88ee7-1b9a-4df9-8d8b-a8a79b768343 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:16.462487 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c88ee7-1b9a-4df9-8d8b-a8a79b768343 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:16.463472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c88ee7-1b9a-4df9-8d8b-a8a79b768343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:01:16.463778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c88ee7-1b9a-4df9-8d8b-a8a79b768343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:16.465376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c88ee7-1b9a-4df9-8d8b-a8a79b768343 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:16.476843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1224/2453] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:01:16.481120 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd1b7e-7f04-4a87-8069-024f5a99a305 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:16.490242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd1b7e-7f04-4a87-8069-024f5a99a305 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:16.495733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15873c20-17f7-4d4b-a111-6470be43ef77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:16.496380 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15873c20-17f7-4d4b-a111-6470be43ef77 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:16.496665 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15873c20-17f7-4d4b-a111-6470be43ef77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:16.496943 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15873c20-17f7-4d4b-a111-6470be43ef77 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:16.497633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15873c20-17f7-4d4b-a111-6470be43ef77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:16.499578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15873c20-17f7-4d4b-a111-6470be43ef77 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:16.525417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1230/2454] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:16 2022] POST /identity/v3/projects => generated 361 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:16.529524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c88ee7-1b9a-4df9-8d8b-a8a79b768343 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:16.539379 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c88ee7-1b9a-4df9-8d8b-a8a79b768343 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:16.595226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc176a0-c306-4421-a8fb-cd6a633184a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:16.595889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc176a0-c306-4421-a8fb-cd6a633184a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:16.596172 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc176a0-c306-4421-a8fb-cd6a633184a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:16.596448 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc176a0-c306-4421-a8fb-cd6a633184a4 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:16.597136 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc176a0-c306-4421-a8fb-cd6a633184a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:16.599057 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc176a0-c306-4421-a8fb-cd6a633184a4 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:16.618252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1225/2455] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:16 2022] POST /identity/v3/users => generated 281 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:16.622887 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15873c20-17f7-4d4b-a111-6470be43ef77 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:16.633295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15873c20-17f7-4d4b-a111-6470be43ef77 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:16.638738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518aa3e7-81ea-4e63-89f7-b1493657776d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:16.639420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aa3e7-81ea-4e63-89f7-b1493657776d admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:16.639704 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aa3e7-81ea-4e63-89f7-b1493657776d admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:16.639980 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aa3e7-81ea-4e63-89f7-b1493657776d admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:16.640757 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518aa3e7-81ea-4e63-89f7-b1493657776d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:16.642424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518aa3e7-81ea-4e63-89f7-b1493657776d admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:16.655171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1231/2456] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:16.658837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc176a0-c306-4421-a8fb-cd6a633184a4 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:16.670889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc176a0-c306-4421-a8fb-cd6a633184a4 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:16.675574 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c9e1d-e141-48a0-aa30-dc5c9a0e3191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:16.676231 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c9e1d-e141-48a0-aa30-dc5c9a0e3191 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:16.676514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c9e1d-e141-48a0-aa30-dc5c9a0e3191 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:16.676790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c9e1d-e141-48a0-aa30-dc5c9a0e3191 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:16.677510 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c9e1d-e141-48a0-aa30-dc5c9a0e3191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:16.679135 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c9e1d-e141-48a0-aa30-dc5c9a0e3191 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:16.690667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1226/2457] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:16.694772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518aa3e7-81ea-4e63-89f7-b1493657776d admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:16.705761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518aa3e7-81ea-4e63-89f7-b1493657776d admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:16.710396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd23b1-e480-4352-a38d-4a976488c4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:16.711298 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd23b1-e480-4352-a38d-4a976488c4c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:16.711577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd23b1-e480-4352-a38d-4a976488c4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:16.711849 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd23b1-e480-4352-a38d-4a976488c4c0 admin admin] PATH_INFO: `/v3/projects/f0b7624fff9d42399a167e3c4a1be184/users/fb428ef44eb042d1a79ddf663c341a79/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:16.727114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd23b1-e480-4352-a38d-4a976488c4c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b7624fff9d42399a167e3c4a1be184, user_id=fb428ef44eb042d1a79ddf663c341a79, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:16.730360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd23b1-e480-4352-a38d-4a976488c4c0 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:16.743577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1232/2458] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:16 2022] PUT /identity/v3/projects/f0b7624fff9d42399a167e3c4a1be184/users/fb428ef44eb042d1a79ddf663c341a79/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:01:16.754778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a393ae-bc4a-44c4-a9ff-5b478d550b13 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:16.755169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a393ae-bc4a-44c4-a9ff-5b478d550b13 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:16.755524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a393ae-bc4a-44c4-a9ff-5b478d550b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:16.784682 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-15a393ae-bc4a-44c4-a9ff-5b478d550b13 None None] MFA Rules not processed for user `fb428ef44eb042d1a79ddf663c341a79`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:16.785984 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-15a393ae-bc4a-44c4-a9ff-5b478d550b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:16.854180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1227/2459] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:16.886948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd23b1-e480-4352-a38d-4a976488c4c0 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:16.888616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1dcd23b1-e480-4352-a38d-4a976488c4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:16.944783 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd23b1-e480-4352-a38d-4a976488c4c0 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:16.949092 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae57be-8e9b-4b82-a3dd-038ad1a56d50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:16.949729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae57be-8e9b-4b82-a3dd-038ad1a56d50 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:16.950015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae57be-8e9b-4b82-a3dd-038ad1a56d50 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:16.950301 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae57be-8e9b-4b82-a3dd-038ad1a56d50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:16.958645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae57be-8e9b-4b82-a3dd-038ad1a56d50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:16.960072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae57be-8e9b-4b82-a3dd-038ad1a56d50 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:16.969796 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1233/2460] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:16.983602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a393ae-bc4a-44c4-a9ff-5b478d550b13 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:17.029209 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:17.047316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a393ae-bc4a-44c4-a9ff-5b478d550b13 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:17.054603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:17.055814 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:17.056082 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:17.056834 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] PATH_INFO: `/v3/users/fb428ef44eb042d1a79ddf663c341a79` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:17.058426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb428ef44eb042d1a79ddf663c341a79)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:17.060381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:17.105475 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb428ef44eb042d1a79ddf663c341a79'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:17.112274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb428ef44eb042d1a79ddf663c341a79'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:17.112936 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb428ef44eb042d1a79ddf663c341a79'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:17.120299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb428ef44eb042d1a79ddf663c341a79'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:17.132031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb428ef44eb042d1a79ddf663c341a79'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:17.138470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1228/2461] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:16 2022] DELETE /identity/v3/users/fb428ef44eb042d1a79ddf663c341a79 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:17.143076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa1cfb-af4c-4d44-8de9-3bc8ad96975a None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:17.143755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa1cfb-af4c-4d44-8de9-3bc8ad96975a None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:17.143953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa1cfb-af4c-4d44-8de9-3bc8ad96975a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:17.166969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-fffa1cfb-af4c-4d44-8de9-3bc8ad96975a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:17.168203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-fffa1cfb-af4c-4d44-8de9-3bc8ad96975a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:17.178389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:17.230504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1234/2462] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:17.241392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:17.242812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:17.301418 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb7546-9f27-41f3-83d2-272b35153399 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:17.305106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1aac0f-ca2f-46d2-b608-e030f358181c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:17.305756 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1aac0f-ca2f-46d2-b608-e030f358181c service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:17.306042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1aac0f-ca2f-46d2-b608-e030f358181c service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:17.306870 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1aac0f-ca2f-46d2-b608-e030f358181c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:17.316129 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1aac0f-ca2f-46d2-b608-e030f358181c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:17.317555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1aac0f-ca2f-46d2-b608-e030f358181c service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:17.327612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1229/2463] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:17.595100 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa1cfb-af4c-4d44-8de9-3bc8ad96975a None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:17.596545 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-fffa1cfb-af4c-4d44-8de9-3bc8ad96975a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:17.609039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa1cfb-af4c-4d44-8de9-3bc8ad96975a None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:17.613625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538214f4-1ee0-48e5-834f-e3dfe4a13b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:17.614491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538214f4-1ee0-48e5-834f-e3dfe4a13b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:17.614772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538214f4-1ee0-48e5-834f-e3dfe4a13b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:17.615116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538214f4-1ee0-48e5-834f-e3dfe4a13b70 admin admin] PATH_INFO: `/v3/projects/f0b7624fff9d42399a167e3c4a1be184` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:17.616303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538214f4-1ee0-48e5-834f-e3dfe4a13b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0b7624fff9d42399a167e3c4a1be184)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:17.618196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538214f4-1ee0-48e5-834f-e3dfe4a13b70 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:17.664237 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-538214f4-1ee0-48e5-834f-e3dfe4a13b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b7624fff9d42399a167e3c4a1be184', 'request_id': 'req-538214f4-1ee0-48e5-834f-e3dfe4a13b70'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:17.671573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-538214f4-1ee0-48e5-834f-e3dfe4a13b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b7624fff9d42399a167e3c4a1be184', 'request_id': 'req-538214f4-1ee0-48e5-834f-e3dfe4a13b70'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:17.678807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-538214f4-1ee0-48e5-834f-e3dfe4a13b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b7624fff9d42399a167e3c4a1be184', 'request_id': 'req-538214f4-1ee0-48e5-834f-e3dfe4a13b70'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:17.691696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-538214f4-1ee0-48e5-834f-e3dfe4a13b70 admin admin] The token cache is being invalidate because project f0b7624fff9d42399a167e3c4a1be184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:17.691988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-538214f4-1ee0-48e5-834f-e3dfe4a13b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:17.692645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-538214f4-1ee0-48e5-834f-e3dfe4a13b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:17.694236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1235/2464] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:17 2022] DELETE /identity/v3/projects/f0b7624fff9d42399a167e3c4a1be184 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:17.699323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae88e1-2c21-4eef-b9bd-5019c4544928 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:17.699798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae88e1-2c21-4eef-b9bd-5019c4544928 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:17.699899 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae88e1-2c21-4eef-b9bd-5019c4544928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:17.717605 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3aae88e1-2c21-4eef-b9bd-5019c4544928 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:17.718790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3aae88e1-2c21-4eef-b9bd-5019c4544928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:17.777269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1230/2465] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:17.784981 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fa503-0150-44b0-9623-fec6ea4d8222 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:17.785307 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fa503-0150-44b0-9623-fec6ea4d8222 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:17.785594 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fa503-0150-44b0-9623-fec6ea4d8222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:17.804262 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-652fa503-0150-44b0-9623-fec6ea4d8222 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:17.805485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-652fa503-0150-44b0-9623-fec6ea4d8222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:17.814152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1236/2466] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:17.819864 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae88e1-2c21-4eef-b9bd-5019c4544928 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:17.829858 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae88e1-2c21-4eef-b9bd-5019c4544928 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:17.834395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32dff2-0ca3-46af-bb18-fced51e1ec20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:17.835068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32dff2-0ca3-46af-bb18-fced51e1ec20 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:17.835348 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32dff2-0ca3-46af-bb18-fced51e1ec20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:17.835624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32dff2-0ca3-46af-bb18-fced51e1ec20 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:17.836349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32dff2-0ca3-46af-bb18-fced51e1ec20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:01:17.836650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32dff2-0ca3-46af-bb18-fced51e1ec20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:17.838878 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32dff2-0ca3-46af-bb18-fced51e1ec20 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:17.840318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:17.852061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1231/2467] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:01:17.856750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fa503-0150-44b0-9623-fec6ea4d8222 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:17.865817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fa503-0150-44b0-9623-fec6ea4d8222 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:17.870639 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1241ca1-d9c0-4094-a756-b63631d6409b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:17.871311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1241ca1-d9c0-4094-a756-b63631d6409b admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:17.871594 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1241ca1-d9c0-4094-a756-b63631d6409b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:17.871869 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1241ca1-d9c0-4094-a756-b63631d6409b admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:17.872558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1241ca1-d9c0-4094-a756-b63631d6409b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:17.874446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1241ca1-d9c0-4094-a756-b63631d6409b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:17.905573 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1237/2468] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:17 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:01:17.907021 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32dff2-0ca3-46af-bb18-fced51e1ec20 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:17.917337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32dff2-0ca3-46af-bb18-fced51e1ec20 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:17.973587 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23787115-70a5-4445-8e58-7cd852aa71b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:17.974268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23787115-70a5-4445-8e58-7cd852aa71b0 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:17.974557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23787115-70a5-4445-8e58-7cd852aa71b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:17.974893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23787115-70a5-4445-8e58-7cd852aa71b0 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:17.975596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23787115-70a5-4445-8e58-7cd852aa71b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:17.977508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23787115-70a5-4445-8e58-7cd852aa71b0 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:17.999675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1232/2469] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:17 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:18.003785 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1241ca1-d9c0-4094-a756-b63631d6409b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:18.016018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1241ca1-d9c0-4094-a756-b63631d6409b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:18.021536 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0ed8e-9ae0-4366-b21b-e13bc8f85c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:18.022208 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0ed8e-9ae0-4366-b21b-e13bc8f85c34 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:18.022490 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0ed8e-9ae0-4366-b21b-e13bc8f85c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:18.022764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0ed8e-9ae0-4366-b21b-e13bc8f85c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:18.023539 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0ed8e-9ae0-4366-b21b-e13bc8f85c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:18.025238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0ed8e-9ae0-4366-b21b-e13bc8f85c34 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:18.036299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1238/2470] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:18.040107 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23787115-70a5-4445-8e58-7cd852aa71b0 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:18.050108 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23787115-70a5-4445-8e58-7cd852aa71b0 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:18.054666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2382acf-50e8-4e3c-b124-73acfbb4e861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:18.055343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2382acf-50e8-4e3c-b124-73acfbb4e861 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:18.055626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2382acf-50e8-4e3c-b124-73acfbb4e861 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:18.055901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2382acf-50e8-4e3c-b124-73acfbb4e861 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:18.057163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2382acf-50e8-4e3c-b124-73acfbb4e861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:18.058828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2382acf-50e8-4e3c-b124-73acfbb4e861 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:18.068640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1233/2471] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:18.073543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0ed8e-9ae0-4366-b21b-e13bc8f85c34 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:18.083428 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0ed8e-9ae0-4366-b21b-e13bc8f85c34 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:18.088237 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93498f82-32de-4b00-a487-13ec179eae73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:18.089116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93498f82-32de-4b00-a487-13ec179eae73 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:18.089404 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93498f82-32de-4b00-a487-13ec179eae73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:18.089714 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93498f82-32de-4b00-a487-13ec179eae73 admin admin] PATH_INFO: `/v3/projects/deb84b43a4af4a51b04e0880b240bffd/users/eda37c5a526d42fba9ab1403d9c11422/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:18.104370 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93498f82-32de-4b00-a487-13ec179eae73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deb84b43a4af4a51b04e0880b240bffd, user_id=eda37c5a526d42fba9ab1403d9c11422, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:18.107090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93498f82-32de-4b00-a487-13ec179eae73 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:18.120418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1239/2472] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:18 2022] PUT /identity/v3/projects/deb84b43a4af4a51b04e0880b240bffd/users/eda37c5a526d42fba9ab1403d9c11422/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:01:18.130180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e10c7-a43d-4f3b-ad25-49b86231250d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:18.130489 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e10c7-a43d-4f3b-ad25-49b86231250d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:18.130763 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e10c7-a43d-4f3b-ad25-49b86231250d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:18.157386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-c20e10c7-a43d-4f3b-ad25-49b86231250d None None] MFA Rules not processed for user `eda37c5a526d42fba9ab1403d9c11422`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:18.158793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c20e10c7-a43d-4f3b-ad25-49b86231250d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:18.221316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1234/2473] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:18.225681 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93498f82-32de-4b00-a487-13ec179eae73 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:18.265844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:18.289331 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93498f82-32de-4b00-a487-13ec179eae73 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:18.294480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055adb73-f1bd-4917-bf9e-397fcba3c3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:18.295158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055adb73-f1bd-4917-bf9e-397fcba3c3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:18.295442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055adb73-f1bd-4917-bf9e-397fcba3c3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:18.295717 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055adb73-f1bd-4917-bf9e-397fcba3c3c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:18.296390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055adb73-f1bd-4917-bf9e-397fcba3c3c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:18.298282 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055adb73-f1bd-4917-bf9e-397fcba3c3c1 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:18.323164 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1240/2474] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:18 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:18.327193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e10c7-a43d-4f3b-ad25-49b86231250d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:18.337679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e10c7-a43d-4f3b-ad25-49b86231250d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:18.392842 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b851a3e-eecf-4299-90c6-771b35896e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:18.393508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b851a3e-eecf-4299-90c6-771b35896e3f admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:18.393795 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b851a3e-eecf-4299-90c6-771b35896e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:18.394076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b851a3e-eecf-4299-90c6-771b35896e3f admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:18.394818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b851a3e-eecf-4299-90c6-771b35896e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:18.396758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b851a3e-eecf-4299-90c6-771b35896e3f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:18.405471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:18.422558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1235/2475] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:18 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:18.424441 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055adb73-f1bd-4917-bf9e-397fcba3c3c1 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:18.435326 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055adb73-f1bd-4917-bf9e-397fcba3c3c1 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:18.439940 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855db22-be66-4219-8dc9-513c844bee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:18.440570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855db22-be66-4219-8dc9-513c844bee4c admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:18.440854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855db22-be66-4219-8dc9-513c844bee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:18.441131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855db22-be66-4219-8dc9-513c844bee4c admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:18.441849 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855db22-be66-4219-8dc9-513c844bee4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:18.443475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855db22-be66-4219-8dc9-513c844bee4c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:18.453864 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1241/2476] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:18.457914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b851a3e-eecf-4299-90c6-771b35896e3f admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:18.468313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b851a3e-eecf-4299-90c6-771b35896e3f admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:18.473699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10a0a1-5084-4adc-89f3-7a50f2c33ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:18.474577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10a0a1-5084-4adc-89f3-7a50f2c33ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:18.474882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10a0a1-5084-4adc-89f3-7a50f2c33ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:18.475179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10a0a1-5084-4adc-89f3-7a50f2c33ac7 admin admin] PATH_INFO: `/v3/projects/2a759e45935f4d3eb644df1d555797b7/users/f3ab98d54dc84758940709da198b5b56/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:18.490593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10a0a1-5084-4adc-89f3-7a50f2c33ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a759e45935f4d3eb644df1d555797b7, user_id=f3ab98d54dc84758940709da198b5b56, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:18.493266 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10a0a1-5084-4adc-89f3-7a50f2c33ac7 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:18.506825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1236/2477] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:18 2022] PUT /identity/v3/projects/2a759e45935f4d3eb644df1d555797b7/users/f3ab98d54dc84758940709da198b5b56/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:18.511270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855db22-be66-4219-8dc9-513c844bee4c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:18.521064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855db22-be66-4219-8dc9-513c844bee4c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:18.577210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f92476-df54-47c7-b3d5-906482b2d5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:18.577861 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f92476-df54-47c7-b3d5-906482b2d5a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:18.578149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f92476-df54-47c7-b3d5-906482b2d5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:18.578433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f92476-df54-47c7-b3d5-906482b2d5a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:18.579205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f92476-df54-47c7-b3d5-906482b2d5a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:18.580847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f92476-df54-47c7-b3d5-906482b2d5a6 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:18.591521 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1242/2478] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:18.595779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10a0a1-5084-4adc-89f3-7a50f2c33ac7 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:18.606698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10a0a1-5084-4adc-89f3-7a50f2c33ac7 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:18.611796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f564e26-5085-40e6-b0a0-c3ac733ce98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:18.613054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f564e26-5085-40e6-b0a0-c3ac733ce98b admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:18.613435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f564e26-5085-40e6-b0a0-c3ac733ce98b admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:18.613791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f564e26-5085-40e6-b0a0-c3ac733ce98b admin admin] PATH_INFO: `/v3/projects/2a759e45935f4d3eb644df1d555797b7/users/f3ab98d54dc84758940709da198b5b56/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:18.616075 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f564e26-5085-40e6-b0a0-c3ac733ce98b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a759e45935f4d3eb644df1d555797b7, user_id=f3ab98d54dc84758940709da198b5b56, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:18.618707 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f564e26-5085-40e6-b0a0-c3ac733ce98b admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:18.629336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1237/2479] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:18 2022] PUT /identity/v3/projects/2a759e45935f4d3eb644df1d555797b7/users/f3ab98d54dc84758940709da198b5b56/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:18.639710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d44c84-96d9-42df-9335-34ed7eb235d4 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:18.640197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d44c84-96d9-42df-9335-34ed7eb235d4 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:18.640306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d44c84-96d9-42df-9335-34ed7eb235d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:18.668046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-53d44c84-96d9-42df-9335-34ed7eb235d4 None None] MFA Rules not processed for user `f3ab98d54dc84758940709da198b5b56`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:18.669282 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-53d44c84-96d9-42df-9335-34ed7eb235d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:18.736734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1243/2480] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:18.747474 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f564e26-5085-40e6-b0a0-c3ac733ce98b admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:18.749191 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-8f564e26-5085-40e6-b0a0-c3ac733ce98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:18.808013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f564e26-5085-40e6-b0a0-c3ac733ce98b admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:18.811598 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0bca9-bb1f-40db-ba5a-8b734c262c53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:18.812237 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0bca9-bb1f-40db-ba5a-8b734c262c53 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:18.812517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0bca9-bb1f-40db-ba5a-8b734c262c53 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:18.812789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0bca9-bb1f-40db-ba5a-8b734c262c53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:18.868141 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0bca9-bb1f-40db-ba5a-8b734c262c53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:18.869725 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0bca9-bb1f-40db-ba5a-8b734c262c53 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:18.878395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1238/2481] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:21.396821 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d44c84-96d9-42df-9335-34ed7eb235d4 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:21.430429 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:21.461105 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d44c84-96d9-42df-9335-34ed7eb235d4 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:21.465830 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:21.466698 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:21.467018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:21.467302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] PATH_INFO: `/v3/users/eda37c5a526d42fba9ab1403d9c11422` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:21.468502 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda37c5a526d42fba9ab1403d9c11422)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:21.470439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:21.525470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda37c5a526d42fba9ab1403d9c11422'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:21.533357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda37c5a526d42fba9ab1403d9c11422'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:21.540391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda37c5a526d42fba9ab1403d9c11422'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:21.541772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda37c5a526d42fba9ab1403d9c11422'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:21.542378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda37c5a526d42fba9ab1403d9c11422'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:21.560239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1244/2482] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:21 2022] DELETE /identity/v3/users/eda37c5a526d42fba9ab1403d9c11422 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:21.564832 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab7d42-31f3-43a9-84a9-926f47ff0b78 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:21.565303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab7d42-31f3-43a9-84a9-926f47ff0b78 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:21.565424 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab7d42-31f3-43a9-84a9-926f47ff0b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:21.584437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-97ab7d42-31f3-43a9-84a9-926f47ff0b78 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:21.585623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-97ab7d42-31f3-43a9-84a9-926f47ff0b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:21.644777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1239/2483] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:21.654962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:21.656400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:21.713180 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f874d3-1839-4a47-b39e-d8c6516a9cb9 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:21.716892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70001e-0c0d-44b0-88f8-24a334a6f627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:21.717545 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70001e-0c0d-44b0-88f8-24a334a6f627 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:21.717831 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70001e-0c0d-44b0-88f8-24a334a6f627 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:21.718106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70001e-0c0d-44b0-88f8-24a334a6f627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:21.727317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70001e-0c0d-44b0-88f8-24a334a6f627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:21.729368 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70001e-0c0d-44b0-88f8-24a334a6f627 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:21.738753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1245/2484] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:22.054886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab7d42-31f3-43a9-84a9-926f47ff0b78 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:22.056768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-97ab7d42-31f3-43a9-84a9-926f47ff0b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:22.069624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab7d42-31f3-43a9-84a9-926f47ff0b78 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:22.077333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:22.078239 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:22.078539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:22.078784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] PATH_INFO: `/v3/projects/deb84b43a4af4a51b04e0880b240bffd` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:22.080054 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deb84b43a4af4a51b04e0880b240bffd)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:22.082003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:22.084649 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:22.133508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deb84b43a4af4a51b04e0880b240bffd', 'request_id': 'req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.140297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deb84b43a4af4a51b04e0880b240bffd', 'request_id': 'req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.148595 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deb84b43a4af4a51b04e0880b240bffd', 'request_id': 'req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.160162 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] The token cache is being invalidate because project deb84b43a4af4a51b04e0880b240bffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:22.160450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.161281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.162844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1240/2485] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:22 2022] DELETE /identity/v3/projects/deb84b43a4af4a51b04e0880b240bffd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:22.166910 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70001e-0c0d-44b0-88f8-24a334a6f627 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:22.177915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70001e-0c0d-44b0-88f8-24a334a6f627 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:22.227049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:22.239281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95caa46b-2879-497a-b515-e204159ae13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:22.240342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caa46b-2879-497a-b515-e204159ae13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:22.240630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caa46b-2879-497a-b515-e204159ae13b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:22.240909 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caa46b-2879-497a-b515-e204159ae13b admin admin] PATH_INFO: `/v3/users/f3ab98d54dc84758940709da198b5b56` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:22.242123 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95caa46b-2879-497a-b515-e204159ae13b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3ab98d54dc84758940709da198b5b56)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:22.244155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95caa46b-2879-497a-b515-e204159ae13b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:22.293492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-95caa46b-2879-497a-b515-e204159ae13b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3ab98d54dc84758940709da198b5b56'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.301027 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-95caa46b-2879-497a-b515-e204159ae13b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3ab98d54dc84758940709da198b5b56'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.308744 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-95caa46b-2879-497a-b515-e204159ae13b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3ab98d54dc84758940709da198b5b56'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.309504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-95caa46b-2879-497a-b515-e204159ae13b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3ab98d54dc84758940709da198b5b56'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.310117 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-95caa46b-2879-497a-b515-e204159ae13b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3ab98d54dc84758940709da198b5b56'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.328195 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1246/2486] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:22 2022] DELETE /identity/v3/users/f3ab98d54dc84758940709da198b5b56 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:22.600009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:22.601462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:22.617102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fcdf7-1ad5-4600-a6b0-4f5a12dd02a0 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:22.676030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:22.676894 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:22.677175 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:22.677446 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea admin admin] PATH_INFO: `/v3/projects/2a759e45935f4d3eb644df1d555797b7` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:22.678607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a759e45935f4d3eb644df1d555797b7)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:22.680696 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:22.727068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a759e45935f4d3eb644df1d555797b7', 'request_id': 'req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.733456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a759e45935f4d3eb644df1d555797b7', 'request_id': 'req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.740664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a759e45935f4d3eb644df1d555797b7', 'request_id': 'req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.751730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea admin admin] The token cache is being invalidate because project 2a759e45935f4d3eb644df1d555797b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:22.752013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.752660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ad3adaf4-522a-4c4d-89f8-9a6b7a2ddcea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:22.754191 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1241/2487] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:22 2022] DELETE /identity/v3/projects/2a759e45935f4d3eb644df1d555797b7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:22.759334 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85abd5-19bc-4540-a848-c7b22168af99 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:22.760005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85abd5-19bc-4540-a848-c7b22168af99 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:22.760005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85abd5-19bc-4540-a848-c7b22168af99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:22.778071 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-1b85abd5-19bc-4540-a848-c7b22168af99 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:22.779297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1b85abd5-19bc-4540-a848-c7b22168af99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:22.838260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1247/2488] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:22.847714 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cdde1-2c35-4de9-9f4f-62fdc0517a5d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:22.848039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cdde1-2c35-4de9-9f4f-62fdc0517a5d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:22.848360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cdde1-2c35-4de9-9f4f-62fdc0517a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:22.867008 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-bc3cdde1-2c35-4de9-9f4f-62fdc0517a5d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:22.868201 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-bc3cdde1-2c35-4de9-9f4f-62fdc0517a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:22.877830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1242/2489] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:22.882325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85abd5-19bc-4540-a848-c7b22168af99 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:22.892224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85abd5-19bc-4540-a848-c7b22168af99 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:22.896799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc2de9-9b99-4dae-992d-53f1ffc1f155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:22.897445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc2de9-9b99-4dae-992d-53f1ffc1f155 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:22.897736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc2de9-9b99-4dae-992d-53f1ffc1f155 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:22.898019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc2de9-9b99-4dae-992d-53f1ffc1f155 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:22.898747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc2de9-9b99-4dae-992d-53f1ffc1f155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:01:22.899097 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc2de9-9b99-4dae-992d-53f1ffc1f155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:22.901010 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc2de9-9b99-4dae-992d-53f1ffc1f155 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:22.912519 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1248/2490] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:01:22.917215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cdde1-2c35-4de9-9f4f-62fdc0517a5d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:22.928727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cdde1-2c35-4de9-9f4f-62fdc0517a5d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:22.934173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49177a-4a4e-488c-9a76-b1b20ca409bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:22.934765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49177a-4a4e-488c-9a76-b1b20ca409bd admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:22.935109 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49177a-4a4e-488c-9a76-b1b20ca409bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:22.935391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49177a-4a4e-488c-9a76-b1b20ca409bd admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:22.936085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49177a-4a4e-488c-9a76-b1b20ca409bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:22.938074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49177a-4a4e-488c-9a76-b1b20ca409bd admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:22.951064 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:22.968038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1243/2491] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:22 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:22.969147 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc2de9-9b99-4dae-992d-53f1ffc1f155 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:22.980322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc2de9-9b99-4dae-992d-53f1ffc1f155 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:23.037217 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d50a6f-179e-428c-918a-55e64aeff8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:23.037888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d50a6f-179e-428c-918a-55e64aeff8ac admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:23.038294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d50a6f-179e-428c-918a-55e64aeff8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:23.038585 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d50a6f-179e-428c-918a-55e64aeff8ac admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:23.039298 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d50a6f-179e-428c-918a-55e64aeff8ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:23.041788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d50a6f-179e-428c-918a-55e64aeff8ac admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:23.048342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:23.064878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1249/2492] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:22 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:23.069098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49177a-4a4e-488c-9a76-b1b20ca409bd admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:23.079843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49177a-4a4e-488c-9a76-b1b20ca409bd admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:23.084529 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7b314-5e98-4fc8-835e-bb1c0e5217f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:23.085202 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7b314-5e98-4fc8-835e-bb1c0e5217f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:23.085477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7b314-5e98-4fc8-835e-bb1c0e5217f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:23.085774 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7b314-5e98-4fc8-835e-bb1c0e5217f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:23.086486 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7b314-5e98-4fc8-835e-bb1c0e5217f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:23.088834 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7b314-5e98-4fc8-835e-bb1c0e5217f4 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:23.099236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1244/2493] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:23.103179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d50a6f-179e-428c-918a-55e64aeff8ac admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:23.113682 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d50a6f-179e-428c-918a-55e64aeff8ac admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:23.118376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a588aa-f706-4818-8869-ac7799862b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:23.119090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a588aa-f706-4818-8869-ac7799862b14 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:23.119384 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a588aa-f706-4818-8869-ac7799862b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:23.119664 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a588aa-f706-4818-8869-ac7799862b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:23.120396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a588aa-f706-4818-8869-ac7799862b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:23.121980 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a588aa-f706-4818-8869-ac7799862b14 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:23.132948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1250/2494] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:23.136986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7b314-5e98-4fc8-835e-bb1c0e5217f4 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:23.146998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7b314-5e98-4fc8-835e-bb1c0e5217f4 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:23.152315 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62846c8-99b2-472c-8ed4-432c5397ef32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:23.153188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62846c8-99b2-472c-8ed4-432c5397ef32 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:23.153473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62846c8-99b2-472c-8ed4-432c5397ef32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:23.153751 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62846c8-99b2-472c-8ed4-432c5397ef32 admin admin] PATH_INFO: `/v3/projects/3c910d5408ed456e98d0c6690dab179f/users/26ce3b646fef41059d34f424cdaabdf6/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:23.168990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62846c8-99b2-472c-8ed4-432c5397ef32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c910d5408ed456e98d0c6690dab179f, user_id=26ce3b646fef41059d34f424cdaabdf6, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:23.171687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62846c8-99b2-472c-8ed4-432c5397ef32 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:23.184806 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1245/2495] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:23 2022] PUT /identity/v3/projects/3c910d5408ed456e98d0c6690dab179f/users/26ce3b646fef41059d34f424cdaabdf6/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:23.194819 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582c7a9-7a91-4815-8576-a85c73f67dbd None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:23.195146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582c7a9-7a91-4815-8576-a85c73f67dbd None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:23.195430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582c7a9-7a91-4815-8576-a85c73f67dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:23.221599 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-a582c7a9-7a91-4815-8576-a85c73f67dbd None None] MFA Rules not processed for user `26ce3b646fef41059d34f424cdaabdf6`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:23.222808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a582c7a9-7a91-4815-8576-a85c73f67dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:23.298186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1251/2496] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:23.308797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62846c8-99b2-472c-8ed4-432c5397ef32 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:23.310514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a62846c8-99b2-472c-8ed4-432c5397ef32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:23.367192 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62846c8-99b2-472c-8ed4-432c5397ef32 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:23.370848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7ca2e-628c-4380-8cc9-fff240b9a6ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:23.371512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7ca2e-628c-4380-8cc9-fff240b9a6ac service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:23.371798 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7ca2e-628c-4380-8cc9-fff240b9a6ac service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:23.372074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7ca2e-628c-4380-8cc9-fff240b9a6ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:23.380320 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7ca2e-628c-4380-8cc9-fff240b9a6ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:23.381997 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7ca2e-628c-4380-8cc9-fff240b9a6ac service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:23.390849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1246/2497] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:26.378493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582c7a9-7a91-4815-8576-a85c73f67dbd None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:26.449886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582c7a9-7a91-4815-8576-a85c73f67dbd None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:26.454374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:26.455295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:26.455606 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:26.455887 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] PATH_INFO: `/v3/users/26ce3b646fef41059d34f424cdaabdf6` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:26.457078 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ce3b646fef41059d34f424cdaabdf6)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:26.459062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:26.506721 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ce3b646fef41059d34f424cdaabdf6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:26.514186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ce3b646fef41059d34f424cdaabdf6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:26.521021 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ce3b646fef41059d34f424cdaabdf6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:26.521650 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ce3b646fef41059d34f424cdaabdf6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:26.522261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ce3b646fef41059d34f424cdaabdf6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:26.527513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:26.541706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1252/2498] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:26 2022] DELETE /identity/v3/users/26ce3b646fef41059d34f424cdaabdf6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:26.546882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6df90-7f97-4dc2-a867-5f3255720e89 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:26.547352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6df90-7f97-4dc2-a867-5f3255720e89 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:26.547467 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6df90-7f97-4dc2-a867-5f3255720e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:26.564806 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-fde6df90-7f97-4dc2-a867-5f3255720e89 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:26.565989 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-fde6df90-7f97-4dc2-a867-5f3255720e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:26.624256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1247/2499] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:26.633966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:26.635438 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:26.695110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d618c-573f-4e49-94b1-62a7a40ff5d0 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:26.698913 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8eb8c7-3eb9-4ec0-ac7e-15d6a7e81864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:26.699564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8eb8c7-3eb9-4ec0-ac7e-15d6a7e81864 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:26.699851 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8eb8c7-3eb9-4ec0-ac7e-15d6a7e81864 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:26.700130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8eb8c7-3eb9-4ec0-ac7e-15d6a7e81864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:26.709876 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8eb8c7-3eb9-4ec0-ac7e-15d6a7e81864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:26.711399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8eb8c7-3eb9-4ec0-ac7e-15d6a7e81864 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:26.722287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1253/2500] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:26.993366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6df90-7f97-4dc2-a867-5f3255720e89 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:26.995074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-fde6df90-7f97-4dc2-a867-5f3255720e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:27.008901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6df90-7f97-4dc2-a867-5f3255720e89 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:27.013803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c85d87-b18b-453b-938c-6eacbd43344a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:27.014801 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c85d87-b18b-453b-938c-6eacbd43344a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:27.015183 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c85d87-b18b-453b-938c-6eacbd43344a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:27.015533 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c85d87-b18b-453b-938c-6eacbd43344a admin admin] PATH_INFO: `/v3/projects/3c910d5408ed456e98d0c6690dab179f` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:27.017010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c85d87-b18b-453b-938c-6eacbd43344a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c910d5408ed456e98d0c6690dab179f)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:27.019413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c85d87-b18b-453b-938c-6eacbd43344a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:27.069640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-10c85d87-b18b-453b-938c-6eacbd43344a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c910d5408ed456e98d0c6690dab179f', 'request_id': 'req-10c85d87-b18b-453b-938c-6eacbd43344a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:27.075741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-10c85d87-b18b-453b-938c-6eacbd43344a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c910d5408ed456e98d0c6690dab179f', 'request_id': 'req-10c85d87-b18b-453b-938c-6eacbd43344a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:27.078433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:27.083583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-10c85d87-b18b-453b-938c-6eacbd43344a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c910d5408ed456e98d0c6690dab179f', 'request_id': 'req-10c85d87-b18b-453b-938c-6eacbd43344a'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:27.095580 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-10c85d87-b18b-453b-938c-6eacbd43344a admin admin] The token cache is being invalidate because project 3c910d5408ed456e98d0c6690dab179f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:27.095868 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-10c85d87-b18b-453b-938c-6eacbd43344a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:27.096494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-10c85d87-b18b-453b-938c-6eacbd43344a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:27.098037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1248/2501] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:26 2022] DELETE /identity/v3/projects/3c910d5408ed456e98d0c6690dab179f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:27.103026 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e6204-28fb-49b7-bd39-0505eebf6e25 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:27.103320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e6204-28fb-49b7-bd39-0505eebf6e25 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:27.103624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e6204-28fb-49b7-bd39-0505eebf6e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:27.121856 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-246e6204-28fb-49b7-bd39-0505eebf6e25 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:27.123092 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-246e6204-28fb-49b7-bd39-0505eebf6e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:27.181403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1254/2502] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:27.189772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3cd2d-450b-4508-8c1b-175b519c7560 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:27.190066 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3cd2d-450b-4508-8c1b-175b519c7560 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:27.190351 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3cd2d-450b-4508-8c1b-175b519c7560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:27.208834 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-0af3cd2d-450b-4508-8c1b-175b519c7560 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:27.210027 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-0af3cd2d-450b-4508-8c1b-175b519c7560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:27.218491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1249/2503] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:27.222723 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e6204-28fb-49b7-bd39-0505eebf6e25 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:27.232475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e6204-28fb-49b7-bd39-0505eebf6e25 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:27.238011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb354c95-ad51-4cc1-8acd-528ac25c64fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:27.238904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb354c95-ad51-4cc1-8acd-528ac25c64fd admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:27.239204 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb354c95-ad51-4cc1-8acd-528ac25c64fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:27.239489 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb354c95-ad51-4cc1-8acd-528ac25c64fd admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:27.240221 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb354c95-ad51-4cc1-8acd-528ac25c64fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:01:27.240533 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb354c95-ad51-4cc1-8acd-528ac25c64fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:27.242141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb354c95-ad51-4cc1-8acd-528ac25c64fd admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:27.254043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1255/2504] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:01:27.258508 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3cd2d-450b-4508-8c1b-175b519c7560 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:27.268292 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3cd2d-450b-4508-8c1b-175b519c7560 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:27.272950 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe207887-17e9-498b-ab1a-7a2a161b2d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:27.273592 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe207887-17e9-498b-ab1a-7a2a161b2d77 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:27.273878 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe207887-17e9-498b-ab1a-7a2a161b2d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:27.274158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe207887-17e9-498b-ab1a-7a2a161b2d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:27.274853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe207887-17e9-498b-ab1a-7a2a161b2d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:27.277544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe207887-17e9-498b-ab1a-7a2a161b2d77 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:27.302806 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1250/2505] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:27 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:27.306927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb354c95-ad51-4cc1-8acd-528ac25c64fd admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:27.317978 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb354c95-ad51-4cc1-8acd-528ac25c64fd admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:27.374269 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688eb3d8-41a7-41c9-b97f-8650b8e60c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:27.374993 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688eb3d8-41a7-41c9-b97f-8650b8e60c24 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:27.375307 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688eb3d8-41a7-41c9-b97f-8650b8e60c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:27.375587 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688eb3d8-41a7-41c9-b97f-8650b8e60c24 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:27.376284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688eb3d8-41a7-41c9-b97f-8650b8e60c24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:27.378277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688eb3d8-41a7-41c9-b97f-8650b8e60c24 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:27.399254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1256/2506] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:27 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:27.404125 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe207887-17e9-498b-ab1a-7a2a161b2d77 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:27.414283 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe207887-17e9-498b-ab1a-7a2a161b2d77 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:27.419052 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce9b52-bf6e-42ca-b300-25f14c4e38d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:27.419706 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce9b52-bf6e-42ca-b300-25f14c4e38d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:27.419986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce9b52-bf6e-42ca-b300-25f14c4e38d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:27.420258 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce9b52-bf6e-42ca-b300-25f14c4e38d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:27.420997 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce9b52-bf6e-42ca-b300-25f14c4e38d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:27.422567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce9b52-bf6e-42ca-b300-25f14c4e38d4 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:27.433774 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1251/2507] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:27.437603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688eb3d8-41a7-41c9-b97f-8650b8e60c24 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:27.447553 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688eb3d8-41a7-41c9-b97f-8650b8e60c24 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:27.452697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486e2ab-5859-4976-8fec-deece827ac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:27.453348 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486e2ab-5859-4976-8fec-deece827ac7d admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:27.453640 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486e2ab-5859-4976-8fec-deece827ac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:27.453921 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486e2ab-5859-4976-8fec-deece827ac7d admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:27.454651 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486e2ab-5859-4976-8fec-deece827ac7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:27.456291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486e2ab-5859-4976-8fec-deece827ac7d admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:27.457534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:27.469966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1257/2508] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:27.473827 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce9b52-bf6e-42ca-b300-25f14c4e38d4 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:27.483642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce9b52-bf6e-42ca-b300-25f14c4e38d4 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:27.488793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a362d9-251b-41a2-b9ac-189d68cdac3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:27.489661 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a362d9-251b-41a2-b9ac-189d68cdac3f admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:27.489939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a362d9-251b-41a2-b9ac-189d68cdac3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:27.490342 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a362d9-251b-41a2-b9ac-189d68cdac3f admin admin] PATH_INFO: `/v3/projects/cc10d4f105854587932c9eda2cc91137/users/cd484bf093dd48d89d4184d9db01b2ef/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:27.504884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a362d9-251b-41a2-b9ac-189d68cdac3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc10d4f105854587932c9eda2cc91137, user_id=cd484bf093dd48d89d4184d9db01b2ef, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:27.507792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a362d9-251b-41a2-b9ac-189d68cdac3f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:27.521363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1252/2509] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:27 2022] PUT /identity/v3/projects/cc10d4f105854587932c9eda2cc91137/users/cd484bf093dd48d89d4184d9db01b2ef/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:27.532675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a532bf-06ee-4f93-b523-bfeb3463b780 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:27.532972 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a532bf-06ee-4f93-b523-bfeb3463b780 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:27.533265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a532bf-06ee-4f93-b523-bfeb3463b780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:27.559987 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-e7a532bf-06ee-4f93-b523-bfeb3463b780 None None] MFA Rules not processed for user `cd484bf093dd48d89d4184d9db01b2ef`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:27.561210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e7a532bf-06ee-4f93-b523-bfeb3463b780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:27.624873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1258/2510] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:27.636544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a362d9-251b-41a2-b9ac-189d68cdac3f admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:27.638241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-62a362d9-251b-41a2-b9ac-189d68cdac3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:27.694376 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a362d9-251b-41a2-b9ac-189d68cdac3f admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:27.698012 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b9619-4b0c-4478-8d08-345002006b2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:27.699345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b9619-4b0c-4478-8d08-345002006b2d service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:27.699651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b9619-4b0c-4478-8d08-345002006b2d service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:27.699946 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b9619-4b0c-4478-8d08-345002006b2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:27.708328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b9619-4b0c-4478-8d08-345002006b2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:27.709745 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b9619-4b0c-4478-8d08-345002006b2d service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:27.719419 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1253/2511] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:35.758200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a532bf-06ee-4f93-b523-bfeb3463b780 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:35.759638 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e7a532bf-06ee-4f93-b523-bfeb3463b780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:35.828819 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a532bf-06ee-4f93-b523-bfeb3463b780 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:35.832662 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140c7a1-b278-4bf2-b57a-97ec9e8a139f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:35.833304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140c7a1-b278-4bf2-b57a-97ec9e8a139f service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:35.833592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140c7a1-b278-4bf2-b57a-97ec9e8a139f service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:35.833870 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140c7a1-b278-4bf2-b57a-97ec9e8a139f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:35.835725 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1140c7a1-b278-4bf2-b57a-97ec9e8a139f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:35.899833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140c7a1-b278-4bf2-b57a-97ec9e8a139f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:35.901261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140c7a1-b278-4bf2-b57a-97ec9e8a139f service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:35.911757 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1259/2512] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:01:37.547817 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b9619-4b0c-4478-8d08-345002006b2d service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:37.559252 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b9619-4b0c-4478-8d08-345002006b2d service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:37.565316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:37.566207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:37.566489 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:37.566766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] PATH_INFO: `/v3/users/cd484bf093dd48d89d4184d9db01b2ef` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:37.568031 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd484bf093dd48d89d4184d9db01b2ef)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:37.570104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:37.628283 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd484bf093dd48d89d4184d9db01b2ef'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:37.640965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd484bf093dd48d89d4184d9db01b2ef'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:37.641918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd484bf093dd48d89d4184d9db01b2ef'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:37.652073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd484bf093dd48d89d4184d9db01b2ef'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:37.665011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:37.666472 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd484bf093dd48d89d4184d9db01b2ef'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:37.674430 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1254/2513] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:37 2022] DELETE /identity/v3/users/cd484bf093dd48d89d4184d9db01b2ef => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:37.679013 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13207b1f-ea23-47de-af73-6638858ab176 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:37.679529 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13207b1f-ea23-47de-af73-6638858ab176 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:37.679778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13207b1f-ea23-47de-af73-6638858ab176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:37.698936 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-13207b1f-ea23-47de-af73-6638858ab176 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:37.701454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-13207b1f-ea23-47de-af73-6638858ab176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:37.736030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=100 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:37.784600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1260/2514] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:37.795974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:37.797519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:37.862168 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8bec5-1a26-481e-9fdd-6b91611a2df0 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:37.865887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a94877-aa23-4f6e-94a8-8fb5a2621ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:37.866699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a94877-aa23-4f6e-94a8-8fb5a2621ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:37.867045 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a94877-aa23-4f6e-94a8-8fb5a2621ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:37.867322 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a94877-aa23-4f6e-94a8-8fb5a2621ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:37.877421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a94877-aa23-4f6e-94a8-8fb5a2621ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:37.878978 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a94877-aa23-4f6e-94a8-8fb5a2621ed2 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:37.888588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1255/2515] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:01:38.177457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13207b1f-ea23-47de-af73-6638858ab176 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:38.179069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-13207b1f-ea23-47de-af73-6638858ab176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:38.212625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13207b1f-ea23-47de-af73-6638858ab176 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:38.217559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:38.218302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:38.219148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:38.219148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0 admin admin] PATH_INFO: `/v3/projects/cc10d4f105854587932c9eda2cc91137` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:38.220104 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc10d4f105854587932c9eda2cc91137)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:38.222445 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:38.274517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc10d4f105854587932c9eda2cc91137', 'request_id': 'req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:38.280770 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc10d4f105854587932c9eda2cc91137', 'request_id': 'req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:38.292678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc10d4f105854587932c9eda2cc91137', 'request_id': 'req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:38.303989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0 admin admin] The token cache is being invalidate because project cc10d4f105854587932c9eda2cc91137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:38.304358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:38.305127 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-4eef2576-4a8b-4c06-ae70-ad39d0bcb0d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:38.306887 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1261/2516] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:38 2022] DELETE /identity/v3/projects/cc10d4f105854587932c9eda2cc91137 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:38.312557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f233d-44ab-4c3a-9cf3-1f5c5a8de206 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:38.312847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f233d-44ab-4c3a-9cf3-1f5c5a8de206 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:38.313128 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f233d-44ab-4c3a-9cf3-1f5c5a8de206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:38.331630 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-b82f233d-44ab-4c3a-9cf3-1f5c5a8de206 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:38.332848 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b82f233d-44ab-4c3a-9cf3-1f5c5a8de206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:38.407939 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1256/2517] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:01:38.416439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77264f-0213-4ec2-b894-bdd4939e0d2d None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:38.417085 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77264f-0213-4ec2-b894-bdd4939e0d2d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:38.417446 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77264f-0213-4ec2-b894-bdd4939e0d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:38.444614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-6e77264f-0213-4ec2-b894-bdd4939e0d2d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:38.445906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-6e77264f-0213-4ec2-b894-bdd4939e0d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:38.455503 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1262/2518] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:38.459904 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f233d-44ab-4c3a-9cf3-1f5c5a8de206 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:38.470211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f233d-44ab-4c3a-9cf3-1f5c5a8de206 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:38.475015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdb739-0914-46ba-a27a-64ac70bb8712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:38.475674 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdb739-0914-46ba-a27a-64ac70bb8712 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:38.475958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdb739-0914-46ba-a27a-64ac70bb8712 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:38.476604 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdb739-0914-46ba-a27a-64ac70bb8712 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:38.476984 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdb739-0914-46ba-a27a-64ac70bb8712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:01:38.477290 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdb739-0914-46ba-a27a-64ac70bb8712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:38.478957 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdb739-0914-46ba-a27a-64ac70bb8712 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:38.491242 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1257/2519] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:01:38.495725 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77264f-0213-4ec2-b894-bdd4939e0d2d None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:38.505190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77264f-0213-4ec2-b894-bdd4939e0d2d None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:38.511415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd7f49-7fb4-4cd6-89c4-413ca94c12eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:38.512804 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd7f49-7fb4-4cd6-89c4-413ca94c12eb admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:38.513095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd7f49-7fb4-4cd6-89c4-413ca94c12eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:38.513375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd7f49-7fb4-4cd6-89c4-413ca94c12eb admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:38.514133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd7f49-7fb4-4cd6-89c4-413ca94c12eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:38.516073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd7f49-7fb4-4cd6-89c4-413ca94c12eb admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:38.547248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1263/2520] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:38 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:01:38.558222 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdb739-0914-46ba-a27a-64ac70bb8712 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:38.575004 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdb739-0914-46ba-a27a-64ac70bb8712 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:38.649546 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587007d4-fa6e-4561-9ad5-c5bc939705d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:38.650228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587007d4-fa6e-4561-9ad5-c5bc939705d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:38.650514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587007d4-fa6e-4561-9ad5-c5bc939705d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:38.650792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587007d4-fa6e-4561-9ad5-c5bc939705d8 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:38.651663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587007d4-fa6e-4561-9ad5-c5bc939705d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:38.653709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587007d4-fa6e-4561-9ad5-c5bc939705d8 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:38.679338 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1258/2521] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:38 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:38.683258 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd7f49-7fb4-4cd6-89c4-413ca94c12eb admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:38.698442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd7f49-7fb4-4cd6-89c4-413ca94c12eb admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:38.705874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a2dbd-8dfd-41f1-af0f-8c6e4e3f701e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:38.706745 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a2dbd-8dfd-41f1-af0f-8c6e4e3f701e admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:38.707181 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a2dbd-8dfd-41f1-af0f-8c6e4e3f701e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:38.708335 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a2dbd-8dfd-41f1-af0f-8c6e4e3f701e admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:38.708738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a2dbd-8dfd-41f1-af0f-8c6e4e3f701e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:38.711225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a2dbd-8dfd-41f1-af0f-8c6e4e3f701e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:38.725588 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1264/2522] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:38.729272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587007d4-fa6e-4561-9ad5-c5bc939705d8 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:38.747556 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587007d4-fa6e-4561-9ad5-c5bc939705d8 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:38.753971 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2d2ea-9500-4177-bd8e-0ba1c33700cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:38.754659 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2d2ea-9500-4177-bd8e-0ba1c33700cf admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:38.754986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2d2ea-9500-4177-bd8e-0ba1c33700cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:38.755271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2d2ea-9500-4177-bd8e-0ba1c33700cf admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:38.756013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2d2ea-9500-4177-bd8e-0ba1c33700cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:38.757871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2d2ea-9500-4177-bd8e-0ba1c33700cf admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:38.769404 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1259/2523] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:01:38.775198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a2dbd-8dfd-41f1-af0f-8c6e4e3f701e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:38.786403 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a2dbd-8dfd-41f1-af0f-8c6e4e3f701e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:38.797399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a4aee-1456-4444-a7f0-bb005969f56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:38.798381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a4aee-1456-4444-a7f0-bb005969f56b admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:38.798747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a4aee-1456-4444-a7f0-bb005969f56b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:38.799330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a4aee-1456-4444-a7f0-bb005969f56b admin admin] PATH_INFO: `/v3/projects/f68358e9d695445490f8209d832a1c13/users/12d3af57f5e34f798336d353a111cefd/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:38.818116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a4aee-1456-4444-a7f0-bb005969f56b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f68358e9d695445490f8209d832a1c13, user_id=12d3af57f5e34f798336d353a111cefd, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:38.822811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a4aee-1456-4444-a7f0-bb005969f56b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:38.842915 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1265/2524] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:38 2022] PUT /identity/v3/projects/f68358e9d695445490f8209d832a1c13/users/12d3af57f5e34f798336d353a111cefd/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:38.847429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da7f59-fb44-4544-a1aa-06c1d5e8fee1 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:38.847726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da7f59-fb44-4544-a1aa-06c1d5e8fee1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:38.848015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da7f59-fb44-4544-a1aa-06c1d5e8fee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:38.856607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:38.869619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-57da7f59-fb44-4544-a1aa-06c1d5e8fee1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:38.871429 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-57da7f59-fb44-4544-a1aa-06c1d5e8fee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:38.888250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a4aee-1456-4444-a7f0-bb005969f56b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:38.891015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-009a4aee-1456-4444-a7f0-bb005969f56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:38.954426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1260/2525] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:38.962099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:38.964325 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da7f59-fb44-4544-a1aa-06c1d5e8fee1 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:38.966536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-57da7f59-fb44-4544-a1aa-06c1d5e8fee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:38.968697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a4aee-1456-4444-a7f0-bb005969f56b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:38.975099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb86b4-df16-48ea-b268-466bc91c9b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:38.976026 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb86b4-df16-48ea-b268-466bc91c9b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:38.976558 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb86b4-df16-48ea-b268-466bc91c9b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:38.977044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb86b4-df16-48ea-b268-466bc91c9b18 admin admin] PATH_INFO: `/v3/users/00b21c3634cc41aca680152740644168` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:38.978550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb86b4-df16-48ea-b268-466bc91c9b18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00b21c3634cc41aca680152740644168)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:38.981197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb86b4-df16-48ea-b268-466bc91c9b18 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:39.028602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f0fb86b4-df16-48ea-b268-466bc91c9b18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00b21c3634cc41aca680152740644168'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:39.028879 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da7f59-fb44-4544-a1aa-06c1d5e8fee1 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:39.032484 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb607eb9-642d-45a7-aa49-601e40640780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:39.033383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb607eb9-642d-45a7-aa49-601e40640780 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:39.033596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb607eb9-642d-45a7-aa49-601e40640780 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:39.033933 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb607eb9-642d-45a7-aa49-601e40640780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:39.038077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f0fb86b4-df16-48ea-b268-466bc91c9b18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00b21c3634cc41aca680152740644168'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:39.045325 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb607eb9-642d-45a7-aa49-601e40640780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:39.046770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb607eb9-642d-45a7-aa49-601e40640780 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:39.047256 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f0fb86b4-df16-48ea-b268-466bc91c9b18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00b21c3634cc41aca680152740644168'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:39.048110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f0fb86b4-df16-48ea-b268-466bc91c9b18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00b21c3634cc41aca680152740644168'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:39.048794 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f0fb86b4-df16-48ea-b268-466bc91c9b18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00b21c3634cc41aca680152740644168'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:39.063359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1261/2526] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:01:39.073307 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1266/2527] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:38 2022] DELETE /identity/v3/users/00b21c3634cc41aca680152740644168 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:39.291253 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb607eb9-642d-45a7-aa49-601e40640780 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:39.312382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb607eb9-642d-45a7-aa49-601e40640780 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:39.385029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:39.385903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:39.386198 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:39.386514 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8 admin admin] PATH_INFO: `/v3/projects/4eed7a893f7b42edaadbf898ebe16cd6` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:39.388766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eed7a893f7b42edaadbf898ebe16cd6)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:39.390742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:39.445715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eed7a893f7b42edaadbf898ebe16cd6', 'request_id': 'req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:39.447473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:39.456921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eed7a893f7b42edaadbf898ebe16cd6', 'request_id': 'req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:39.465433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eed7a893f7b42edaadbf898ebe16cd6', 'request_id': 'req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:39.476996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8 admin admin] The token cache is being invalidate because project 4eed7a893f7b42edaadbf898ebe16cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:39.477802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:39.478576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-25e238f3-f3bb-4cab-aa0c-6a717a9383c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:39.484706 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1262/2528] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:39 2022] DELETE /identity/v3/projects/4eed7a893f7b42edaadbf898ebe16cd6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:39.487284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff92bc-14c4-4806-9774-1029a9cd04f1 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:39.488019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff92bc-14c4-4806-9774-1029a9cd04f1 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:39.488511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff92bc-14c4-4806-9774-1029a9cd04f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:39.513810 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-7aff92bc-14c4-4806-9774-1029a9cd04f1 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:39.515176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-7aff92bc-14c4-4806-9774-1029a9cd04f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:39.578073 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1267/2529] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:39.585913 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0a660-2c31-4088-90de-780b6826a7a2 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:39.586440 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0a660-2c31-4088-90de-780b6826a7a2 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:39.586568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0a660-2c31-4088-90de-780b6826a7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:39.608074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-d7f0a660-2c31-4088-90de-780b6826a7a2 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:39.609349 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d7f0a660-2c31-4088-90de-780b6826a7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:39.617437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1263/2530] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:39.621815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff92bc-14c4-4806-9774-1029a9cd04f1 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:39.631810 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff92bc-14c4-4806-9774-1029a9cd04f1 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:39.636355 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3476af7-170a-456a-b618-0ec184626a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:39.637155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3476af7-170a-456a-b618-0ec184626a94 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:39.637494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3476af7-170a-456a-b618-0ec184626a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:39.637803 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3476af7-170a-456a-b618-0ec184626a94 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:39.638536 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3476af7-170a-456a-b618-0ec184626a94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:01:39.638883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3476af7-170a-456a-b618-0ec184626a94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:39.640759 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3476af7-170a-456a-b618-0ec184626a94 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:39.656962 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1268/2531] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:01:39.658041 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0a660-2c31-4088-90de-780b6826a7a2 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:39.667269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0a660-2c31-4088-90de-780b6826a7a2 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:39.672644 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e97d92-39d3-4bb0-9e97-6a0673b03a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:39.673293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e97d92-39d3-4bb0-9e97-6a0673b03a01 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:39.673582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e97d92-39d3-4bb0-9e97-6a0673b03a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:39.673900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e97d92-39d3-4bb0-9e97-6a0673b03a01 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:39.674589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e97d92-39d3-4bb0-9e97-6a0673b03a01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:39.676703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e97d92-39d3-4bb0-9e97-6a0673b03a01 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:39.703667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1264/2532] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:39 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:01:39.707743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3476af7-170a-456a-b618-0ec184626a94 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:39.721197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3476af7-170a-456a-b618-0ec184626a94 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:39.784030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331373e-86f7-426e-b1bb-b2959c7a3b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:39.785440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331373e-86f7-426e-b1bb-b2959c7a3b3c admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:39.785440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331373e-86f7-426e-b1bb-b2959c7a3b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:39.785440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331373e-86f7-426e-b1bb-b2959c7a3b3c admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:39.785932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331373e-86f7-426e-b1bb-b2959c7a3b3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:39.787931 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331373e-86f7-426e-b1bb-b2959c7a3b3c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:39.790964 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c1e5e-59ff-4938-9f41-3d19927b8f59 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:39.791247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c1e5e-59ff-4938-9f41-3d19927b8f59 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:39.791582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c1e5e-59ff-4938-9f41-3d19927b8f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:39.801070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:39.820712 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1269/2533] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:39 2022] POST /identity/v3/users => generated 303 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:39.822566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331373e-86f7-426e-b1bb-b2959c7a3b3c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:39.824599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-443c1e5e-59ff-4938-9f41-3d19927b8f59 None None] MFA Rules not processed for user `12d3af57f5e34f798336d353a111cefd`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:39.825769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-443c1e5e-59ff-4938-9f41-3d19927b8f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:39.835401 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331373e-86f7-426e-b1bb-b2959c7a3b3c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:39.841387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cf2f2-7d36-4559-b1d6-591807b305ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:39.842058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf2f2-7d36-4559-b1d6-591807b305ce admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:39.842350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf2f2-7d36-4559-b1d6-591807b305ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:39.842630 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf2f2-7d36-4559-b1d6-591807b305ce admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:39.843400 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cf2f2-7d36-4559-b1d6-591807b305ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:39.845040 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cf2f2-7d36-4559-b1d6-591807b305ce admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:39.858753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1270/2534] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:39.863258 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cf2f2-7d36-4559-b1d6-591807b305ce admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:39.877042 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cf2f2-7d36-4559-b1d6-591807b305ce admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:39.882801 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27914aa8-1f7d-4d7a-b3b9-c0cf55022b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:39.883502 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27914aa8-1f7d-4d7a-b3b9-c0cf55022b6f admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:39.883780 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27914aa8-1f7d-4d7a-b3b9-c0cf55022b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:39.884055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27914aa8-1f7d-4d7a-b3b9-c0cf55022b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:39.884787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27914aa8-1f7d-4d7a-b3b9-c0cf55022b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:39.886393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27914aa8-1f7d-4d7a-b3b9-c0cf55022b6f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:39.902081 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1271/2535] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:39.902374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1265/2536] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:01:39.904570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27914aa8-1f7d-4d7a-b3b9-c0cf55022b6f admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:39.912468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c1e5e-59ff-4938-9f41-3d19927b8f59 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:39.914045 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-443c1e5e-59ff-4938-9f41-3d19927b8f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:39.915294 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27914aa8-1f7d-4d7a-b3b9-c0cf55022b6f admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:39.920148 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57b0b7-2738-4dcf-a94d-d7da0b310b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:39.921106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57b0b7-2738-4dcf-a94d-d7da0b310b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:39.921453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57b0b7-2738-4dcf-a94d-d7da0b310b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:39.921788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57b0b7-2738-4dcf-a94d-d7da0b310b1f admin admin] PATH_INFO: `/v3/projects/53b53a3cfa1247e98db5447b44d37333/users/d661e0f1262740fbbff4d2357d30657c/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:39.943059 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57b0b7-2738-4dcf-a94d-d7da0b310b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=53b53a3cfa1247e98db5447b44d37333, user_id=d661e0f1262740fbbff4d2357d30657c, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:39.946302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57b0b7-2738-4dcf-a94d-d7da0b310b1f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:39.965979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1272/2537] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:39 2022] PUT /identity/v3/projects/53b53a3cfa1247e98db5447b44d37333/users/d661e0f1262740fbbff4d2357d30657c/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:01:39.975363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c1e5e-59ff-4938-9f41-3d19927b8f59 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:39.976696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16ac38-8e2c-4975-badd-d5566e4d8395 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:39.977094 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16ac38-8e2c-4975-badd-d5566e4d8395 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:39.977448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16ac38-8e2c-4975-badd-d5566e4d8395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:40.006794 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ee16ac38-8e2c-4975-badd-d5566e4d8395 None None] MFA Rules not processed for user `d661e0f1262740fbbff4d2357d30657c`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:40.008048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ee16ac38-8e2c-4975-badd-d5566e4d8395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:40.027932 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607bba8-9719-4c7f-bdb0-1efbc4113379 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:40.028672 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607bba8-9719-4c7f-bdb0-1efbc4113379 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:40.028949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607bba8-9719-4c7f-bdb0-1efbc4113379 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:40.029221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607bba8-9719-4c7f-bdb0-1efbc4113379 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:40.037976 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:40.073870 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1273/2538] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:39 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:01:40.078816 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16ac38-8e2c-4975-badd-d5566e4d8395 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:40.090664 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607bba8-9719-4c7f-bdb0-1efbc4113379 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:40.092295 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607bba8-9719-4c7f-bdb0-1efbc4113379 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:40.101448 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1266/2539] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:01:40.140866 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16ac38-8e2c-4975-badd-d5566e4d8395 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:40.145495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceda8e4-c24b-42ad-8a4b-85342e52af53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:40.146145 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceda8e4-c24b-42ad-8a4b-85342e52af53 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:40.146453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceda8e4-c24b-42ad-8a4b-85342e52af53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:40.146732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceda8e4-c24b-42ad-8a4b-85342e52af53 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:40.147450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceda8e4-c24b-42ad-8a4b-85342e52af53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:40.149322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceda8e4-c24b-42ad-8a4b-85342e52af53 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:40.176360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1274/2540] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:40 2022] POST /identity/v3/projects => generated 403 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:40.180477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607bba8-9719-4c7f-bdb0-1efbc4113379 service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:40.191576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607bba8-9719-4c7f-bdb0-1efbc4113379 service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:40.253360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c58801-909f-4afc-9872-e374d62d1356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:40.254196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c58801-909f-4afc-9872-e374d62d1356 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:40.254611 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c58801-909f-4afc-9872-e374d62d1356 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:40.255073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c58801-909f-4afc-9872-e374d62d1356 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:40.256169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c58801-909f-4afc-9872-e374d62d1356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:40.258132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c58801-909f-4afc-9872-e374d62d1356 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:40.282039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1267/2541] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:40 2022] POST /identity/v3/users => generated 308 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:40.283511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceda8e4-c24b-42ad-8a4b-85342e52af53 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:40.295357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceda8e4-c24b-42ad-8a4b-85342e52af53 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:40.300534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dd0bd-8782-4102-8480-8402d25537f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:40.301198 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd0bd-8782-4102-8480-8402d25537f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:40.301488 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd0bd-8782-4102-8480-8402d25537f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:40.301825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd0bd-8782-4102-8480-8402d25537f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:40.302565 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dd0bd-8782-4102-8480-8402d25537f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:40.304211 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dd0bd-8782-4102-8480-8402d25537f2 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:40.318348 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1275/2542] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:40.322597 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c58801-909f-4afc-9872-e374d62d1356 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:40.334134 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c58801-909f-4afc-9872-e374d62d1356 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:40.338842 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3af9d2-ab22-4814-9ef0-a66d3cf60662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:40.339760 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3af9d2-ab22-4814-9ef0-a66d3cf60662 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:40.341447 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3af9d2-ab22-4814-9ef0-a66d3cf60662 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:40.341745 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3af9d2-ab22-4814-9ef0-a66d3cf60662 admin admin] PATH_INFO: `/v3/projects/b7f3f230bb3e4048a71fc9f69fe0d0f5/users/aaf63ac3982a4a1fbac430accd2e03a5/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:40.357623 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3af9d2-ab22-4814-9ef0-a66d3cf60662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f3f230bb3e4048a71fc9f69fe0d0f5, user_id=aaf63ac3982a4a1fbac430accd2e03a5, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:40.360012 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3af9d2-ab22-4814-9ef0-a66d3cf60662 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:40.375752 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1268/2543] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:40 2022] PUT /identity/v3/projects/b7f3f230bb3e4048a71fc9f69fe0d0f5/users/aaf63ac3982a4a1fbac430accd2e03a5/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:40.380000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dd0bd-8782-4102-8480-8402d25537f2 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:40.399192 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dd0bd-8782-4102-8480-8402d25537f2 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:40.407689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:40.473212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30829262-13a4-49a3-9c33-d66381a5d3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:40.473888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829262-13a4-49a3-9c33-d66381a5d3dc admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:40.474177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829262-13a4-49a3-9c33-d66381a5d3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:40.474457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829262-13a4-49a3-9c33-d66381a5d3dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:40.475306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30829262-13a4-49a3-9c33-d66381a5d3dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:40.477456 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30829262-13a4-49a3-9c33-d66381a5d3dc admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:40.488360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1276/2544] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:40.492800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3af9d2-ab22-4814-9ef0-a66d3cf60662 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:40.503700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3af9d2-ab22-4814-9ef0-a66d3cf60662 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:40.508436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74154e-1a44-427c-953e-96ff269f3c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:40.509334 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74154e-1a44-427c-953e-96ff269f3c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:40.509617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74154e-1a44-427c-953e-96ff269f3c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:40.509891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74154e-1a44-427c-953e-96ff269f3c2a admin admin] PATH_INFO: `/v3/projects/b7f3f230bb3e4048a71fc9f69fe0d0f5/users/aaf63ac3982a4a1fbac430accd2e03a5/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:40.512600 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74154e-1a44-427c-953e-96ff269f3c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f3f230bb3e4048a71fc9f69fe0d0f5, user_id=aaf63ac3982a4a1fbac430accd2e03a5, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:40.515298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74154e-1a44-427c-953e-96ff269f3c2a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:40.525215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1269/2545] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:40 2022] PUT /identity/v3/projects/b7f3f230bb3e4048a71fc9f69fe0d0f5/users/aaf63ac3982a4a1fbac430accd2e03a5/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:40.535431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22331a18-1951-4bc6-a510-c602d9e30744 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:40.535841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22331a18-1951-4bc6-a510-c602d9e30744 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:40.536140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22331a18-1951-4bc6-a510-c602d9e30744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:40.563935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-22331a18-1951-4bc6-a510-c602d9e30744 None None] MFA Rules not processed for user `aaf63ac3982a4a1fbac430accd2e03a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:40.565153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-22331a18-1951-4bc6-a510-c602d9e30744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:40.632791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1277/2546] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:40 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:40.643765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74154e-1a44-427c-953e-96ff269f3c2a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:40.698434 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74154e-1a44-427c-953e-96ff269f3c2a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:40.702282 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15e467-b0ac-4ee2-be6e-c2335efd3e75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:40.702954 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15e467-b0ac-4ee2-be6e-c2335efd3e75 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:40.703242 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15e467-b0ac-4ee2-be6e-c2335efd3e75 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:40.703521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15e467-b0ac-4ee2-be6e-c2335efd3e75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:40.712486 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15e467-b0ac-4ee2-be6e-c2335efd3e75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:40.713900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15e467-b0ac-4ee2-be6e-c2335efd3e75 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:40.723385 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1270/2547] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:40 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:40.880555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22331a18-1951-4bc6-a510-c602d9e30744 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:40.943102 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22331a18-1951-4bc6-a510-c602d9e30744 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:40.947728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:40.948589 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:40.948878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:40.949171 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] PATH_INFO: `/v3/users/d661e0f1262740fbbff4d2357d30657c` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:40.950381 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d661e0f1262740fbbff4d2357d30657c)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:40.952362 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:40.986409 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:41.000195 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd661e0f1262740fbbff4d2357d30657c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.007901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd661e0f1262740fbbff4d2357d30657c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.015015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd661e0f1262740fbbff4d2357d30657c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.015697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd661e0f1262740fbbff4d2357d30657c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.016350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd661e0f1262740fbbff4d2357d30657c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.034023 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1278/2548] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:40 2022] DELETE /identity/v3/users/d661e0f1262740fbbff4d2357d30657c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:41.038488 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410394a-6d79-42c4-b1cf-6bce71331a1e None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:41.038779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410394a-6d79-42c4-b1cf-6bce71331a1e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:41.039100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410394a-6d79-42c4-b1cf-6bce71331a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:41.046214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:41.058006 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-8410394a-6d79-42c4-b1cf-6bce71331a1e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:41.059400 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-8410394a-6d79-42c4-b1cf-6bce71331a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:41.118110 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1271/2549] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:41.127974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:41.129389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:41.187124 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da14754-12c8-4dc7-98d7-510ed26164f0 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:41.190748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e144e6-a572-4873-bec7-f3a00568fb23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:41.191417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e144e6-a572-4873-bec7-f3a00568fb23 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:41.191701 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e144e6-a572-4873-bec7-f3a00568fb23 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:41.191977 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e144e6-a572-4873-bec7-f3a00568fb23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:41.200969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e144e6-a572-4873-bec7-f3a00568fb23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:41.202393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e144e6-a572-4873-bec7-f3a00568fb23 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:41.211825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1279/2550] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:41.475296 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa064-6f20-4332-91b6-9c1787ac2f1d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:41.475618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa064-6f20-4332-91b6-9c1787ac2f1d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:41.475936 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa064-6f20-4332-91b6-9c1787ac2f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:41.488535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e144e6-a572-4873-bec7-f3a00568fb23 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:41.489952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c9e144e6-a572-4873-bec7-f3a00568fb23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:41.496551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-758fa064-6f20-4332-91b6-9c1787ac2f1d None None] MFA Rules not processed for user `12d3af57f5e34f798336d353a111cefd`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:41.498278 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-758fa064-6f20-4332-91b6-9c1787ac2f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:41.515432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e144e6-a572-4873-bec7-f3a00568fb23 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:41.520706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:41.521617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:41.521899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:41.522173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] PATH_INFO: `/v3/projects/53b53a3cfa1247e98db5447b44d37333` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:41.523523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53b53a3cfa1247e98db5447b44d37333)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:41.525569 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:41.567729 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1272/2551] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:41.576668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53b53a3cfa1247e98db5447b44d37333', 'request_id': 'req-32f7e0a9-d74e-4a54-a793-9bf72de9f138'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.582194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53b53a3cfa1247e98db5447b44d37333', 'request_id': 'req-32f7e0a9-d74e-4a54-a793-9bf72de9f138'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.589552 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53b53a3cfa1247e98db5447b44d37333', 'request_id': 'req-32f7e0a9-d74e-4a54-a793-9bf72de9f138'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.601289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] The token cache is being invalidate because project 53b53a3cfa1247e98db5447b44d37333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:41.601579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.602248 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.603817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1280/2552] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:41 2022] DELETE /identity/v3/projects/53b53a3cfa1247e98db5447b44d37333 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:01:41.608398 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fa064-6f20-4332-91b6-9c1787ac2f1d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:41.619070 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fa064-6f20-4332-91b6-9c1787ac2f1d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:41.677610 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:41.678485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:41.678766 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:41.679074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] PATH_INFO: `/v3/users/aaf63ac3982a4a1fbac430accd2e03a5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:41.680259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaf63ac3982a4a1fbac430accd2e03a5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:41.682170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:41.733219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaf63ac3982a4a1fbac430accd2e03a5'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.737957 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:41.742065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaf63ac3982a4a1fbac430accd2e03a5'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.742941 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaf63ac3982a4a1fbac430accd2e03a5'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.752219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaf63ac3982a4a1fbac430accd2e03a5'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.765980 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaf63ac3982a4a1fbac430accd2e03a5'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:41.772420 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1273/2553] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:41 2022] DELETE /identity/v3/users/aaf63ac3982a4a1fbac430accd2e03a5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:01:41.816982 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:41.818482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:41.867208 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:41.887791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7e0a9-d74e-4a54-a793-9bf72de9f138 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:41.944097 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14eb7d7-a24b-4b8d-ab94-814e8ba0200b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:41.944761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14eb7d7-a24b-4b8d-ab94-814e8ba0200b service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:41.945054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14eb7d7-a24b-4b8d-ab94-814e8ba0200b service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:41.945343 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14eb7d7-a24b-4b8d-ab94-814e8ba0200b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:41.947176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d14eb7d7-a24b-4b8d-ab94-814e8ba0200b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:41.957798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14eb7d7-a24b-4b8d-ab94-814e8ba0200b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:41.959286 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14eb7d7-a24b-4b8d-ab94-814e8ba0200b service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:41.968410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1281/2554] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:42.393306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:42.394768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:42.467907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45166f92-2170-48d8-b143-67bea76e1c76 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:42.475200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb902356-6d7a-4992-8824-7c89bf174c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:42.476073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb902356-6d7a-4992-8824-7c89bf174c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:42.476363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb902356-6d7a-4992-8824-7c89bf174c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:42.476642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb902356-6d7a-4992-8824-7c89bf174c63 admin admin] PATH_INFO: `/v3/projects/b7f3f230bb3e4048a71fc9f69fe0d0f5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:42.477879 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb902356-6d7a-4992-8824-7c89bf174c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7f3f230bb3e4048a71fc9f69fe0d0f5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:42.479865 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb902356-6d7a-4992-8824-7c89bf174c63 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:42.529960 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-cb902356-6d7a-4992-8824-7c89bf174c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f3f230bb3e4048a71fc9f69fe0d0f5', 'request_id': 'req-cb902356-6d7a-4992-8824-7c89bf174c63'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:42.536568 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-cb902356-6d7a-4992-8824-7c89bf174c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f3f230bb3e4048a71fc9f69fe0d0f5', 'request_id': 'req-cb902356-6d7a-4992-8824-7c89bf174c63'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:42.551852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-cb902356-6d7a-4992-8824-7c89bf174c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f3f230bb3e4048a71fc9f69fe0d0f5', 'request_id': 'req-cb902356-6d7a-4992-8824-7c89bf174c63'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:42.566441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-cb902356-6d7a-4992-8824-7c89bf174c63 admin admin] The token cache is being invalidate because project b7f3f230bb3e4048a71fc9f69fe0d0f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:42.566441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-cb902356-6d7a-4992-8824-7c89bf174c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:42.567233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-cb902356-6d7a-4992-8824-7c89bf174c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:42.569226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1274/2555] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:42 2022] DELETE /identity/v3/projects/b7f3f230bb3e4048a71fc9f69fe0d0f5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:42.575444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b8a30-caae-4b07-8fe9-cefad5bbec8b None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:42.579113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b8a30-caae-4b07-8fe9-cefad5bbec8b None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:42.579555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b8a30-caae-4b07-8fe9-cefad5bbec8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:42.601355 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-c55b8a30-caae-4b07-8fe9-cefad5bbec8b None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:42.602758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c55b8a30-caae-4b07-8fe9-cefad5bbec8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:42.676383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1282/2556] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:42.685150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dfcc0-aaea-48c2-9c5d-f8965870f138 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:42.685652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dfcc0-aaea-48c2-9c5d-f8965870f138 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:42.685747 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dfcc0-aaea-48c2-9c5d-f8965870f138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:42.705449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-018dfcc0-aaea-48c2-9c5d-f8965870f138 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:42.706965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-018dfcc0-aaea-48c2-9c5d-f8965870f138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:42.718895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1275/2557] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:42.720904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b8a30-caae-4b07-8fe9-cefad5bbec8b None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:42.732524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b8a30-caae-4b07-8fe9-cefad5bbec8b None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:42.737345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fb178-b841-4797-8d03-c809cdb27f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:42.737993 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fb178-b841-4797-8d03-c809cdb27f33 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:42.738273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fb178-b841-4797-8d03-c809cdb27f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:42.738546 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fb178-b841-4797-8d03-c809cdb27f33 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:42.739313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fb178-b841-4797-8d03-c809cdb27f33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:01:42.739618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fb178-b841-4797-8d03-c809cdb27f33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:42.741200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fb178-b841-4797-8d03-c809cdb27f33 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:42.754976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1283/2558] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:01:42.760007 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018dfcc0-aaea-48c2-9c5d-f8965870f138 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:42.769578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018dfcc0-aaea-48c2-9c5d-f8965870f138 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:42.774146 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab009c58-10fb-45ad-8e20-c14512540b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:42.774812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab009c58-10fb-45ad-8e20-c14512540b8d admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:42.775122 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab009c58-10fb-45ad-8e20-c14512540b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:42.775400 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab009c58-10fb-45ad-8e20-c14512540b8d admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:42.776089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab009c58-10fb-45ad-8e20-c14512540b8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:42.778646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab009c58-10fb-45ad-8e20-c14512540b8d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:42.809791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1276/2559] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:42 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:42.811156 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fb178-b841-4797-8d03-c809cdb27f33 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:42.823000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fb178-b841-4797-8d03-c809cdb27f33 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:42.891131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ce2de-e575-4b49-94d6-9753f6dfd876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:42.891811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ce2de-e575-4b49-94d6-9753f6dfd876 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:42.892099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ce2de-e575-4b49-94d6-9753f6dfd876 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:42.892492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ce2de-e575-4b49-94d6-9753f6dfd876 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:42.893471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ce2de-e575-4b49-94d6-9753f6dfd876 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:42.896415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ce2de-e575-4b49-94d6-9753f6dfd876 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:42.919831 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1284/2560] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:42 2022] POST /identity/v3/users => generated 302 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:42.924043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab009c58-10fb-45ad-8e20-c14512540b8d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:42.935934 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab009c58-10fb-45ad-8e20-c14512540b8d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:42.940900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3088056-401f-4e9f-9f1f-586751b5cf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:42.941559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3088056-401f-4e9f-9f1f-586751b5cf44 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:42.941840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3088056-401f-4e9f-9f1f-586751b5cf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:42.942111 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3088056-401f-4e9f-9f1f-586751b5cf44 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:42.942879 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3088056-401f-4e9f-9f1f-586751b5cf44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:42.944468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3088056-401f-4e9f-9f1f-586751b5cf44 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:42.956515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1277/2561] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:01:42.960501 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ce2de-e575-4b49-94d6-9753f6dfd876 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:42.972128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ce2de-e575-4b49-94d6-9753f6dfd876 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:42.976828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7eb4fd-3562-4b94-aff2-68ba97c4b397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:42.977495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eb4fd-3562-4b94-aff2-68ba97c4b397 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:42.977782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eb4fd-3562-4b94-aff2-68ba97c4b397 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:42.978066 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eb4fd-3562-4b94-aff2-68ba97c4b397 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:42.978816 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7eb4fd-3562-4b94-aff2-68ba97c4b397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:42.980423 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7eb4fd-3562-4b94-aff2-68ba97c4b397 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:42.986291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:42.995668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1285/2562] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:42.996474 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3088056-401f-4e9f-9f1f-586751b5cf44 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:43.007537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3088056-401f-4e9f-9f1f-586751b5cf44 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:43.012895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f2a48-b438-4d82-bd1e-f979d1453810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:43.013822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f2a48-b438-4d82-bd1e-f979d1453810 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:43.014105 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f2a48-b438-4d82-bd1e-f979d1453810 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:43.014377 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f2a48-b438-4d82-bd1e-f979d1453810 admin admin] PATH_INFO: `/v3/projects/09301e0029c04454862ab0b7634458ab/users/7cdee476b12247178137555c3c555e57/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:43.031647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f2a48-b438-4d82-bd1e-f979d1453810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09301e0029c04454862ab0b7634458ab, user_id=7cdee476b12247178137555c3c555e57, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:43.034317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f2a48-b438-4d82-bd1e-f979d1453810 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:43.037550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:43.049857 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1278/2563] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:42 2022] PUT /identity/v3/projects/09301e0029c04454862ab0b7634458ab/users/7cdee476b12247178137555c3c555e57/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:43.055010 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1226ac9-7152-46f6-aac0-588ef1ee84ca None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:43.055395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1226ac9-7152-46f6-aac0-588ef1ee84ca None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:43.055680 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1226ac9-7152-46f6-aac0-588ef1ee84ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:43.073261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-d1226ac9-7152-46f6-aac0-588ef1ee84ca None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:43.075047 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d1226ac9-7152-46f6-aac0-588ef1ee84ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:43.135377 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1286/2564] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:43.146949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f2a48-b438-4d82-bd1e-f979d1453810 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:43.149703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-526f2a48-b438-4d82-bd1e-f979d1453810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:43.212780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f2a48-b438-4d82-bd1e-f979d1453810 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:43.217463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede845-e189-4868-8ce4-b22e220eea38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:43.218113 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede845-e189-4868-8ce4-b22e220eea38 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:43.218421 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede845-e189-4868-8ce4-b22e220eea38 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:43.218694 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede845-e189-4868-8ce4-b22e220eea38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:43.229860 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ede845-e189-4868-8ce4-b22e220eea38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:43.231601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ede845-e189-4868-8ce4-b22e220eea38 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:43.241828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1279/2565] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:44.423807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f9236-d682-48d3-8399-6b8c01e0679d None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:44.424306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f9236-d682-48d3-8399-6b8c01e0679d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:44.424404 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f9236-d682-48d3-8399-6b8c01e0679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:44.455780 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-bd0f9236-d682-48d3-8399-6b8c01e0679d None None] MFA Rules not processed for user `7cdee476b12247178137555c3c555e57`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:44.457206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-bd0f9236-d682-48d3-8399-6b8c01e0679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:44.535110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1287/2566] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:44.540269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede845-e189-4868-8ce4-b22e220eea38 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:44.550001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede845-e189-4868-8ce4-b22e220eea38 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:44.556471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207364f7-3ba5-4e94-a5b2-c8284ae1f626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:44.557121 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207364f7-3ba5-4e94-a5b2-c8284ae1f626 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:44.557400 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207364f7-3ba5-4e94-a5b2-c8284ae1f626 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:44.557675 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207364f7-3ba5-4e94-a5b2-c8284ae1f626 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:44.558357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207364f7-3ba5-4e94-a5b2-c8284ae1f626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:44.560452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207364f7-3ba5-4e94-a5b2-c8284ae1f626 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:44.590529 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1280/2567] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:44 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:44.594721 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f9236-d682-48d3-8399-6b8c01e0679d None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:44.607197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f9236-d682-48d3-8399-6b8c01e0679d None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:44.666601 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bc133-4013-4191-95b9-a58ac2f26987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:44.667224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bc133-4013-4191-95b9-a58ac2f26987 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:44.667514 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bc133-4013-4191-95b9-a58ac2f26987 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:44.667820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bc133-4013-4191-95b9-a58ac2f26987 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:44.668861 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bc133-4013-4191-95b9-a58ac2f26987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:44.671036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bc133-4013-4191-95b9-a58ac2f26987 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:44.694273 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1288/2568] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:44 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:01:44.695673 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207364f7-3ba5-4e94-a5b2-c8284ae1f626 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:44.706771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207364f7-3ba5-4e94-a5b2-c8284ae1f626 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:44.711503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35296c-21f7-44af-aebb-b64173585c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:44.712165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35296c-21f7-44af-aebb-b64173585c52 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:44.712449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35296c-21f7-44af-aebb-b64173585c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:44.712726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35296c-21f7-44af-aebb-b64173585c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:44.713436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35296c-21f7-44af-aebb-b64173585c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:44.715193 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35296c-21f7-44af-aebb-b64173585c52 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:44.725941 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1281/2569] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:44.731550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bc133-4013-4191-95b9-a58ac2f26987 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:44.741255 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bc133-4013-4191-95b9-a58ac2f26987 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:44.746800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a48b6b-df64-4e93-8b10-51a0e97f9441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:44.747858 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a48b6b-df64-4e93-8b10-51a0e97f9441 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:44.748208 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a48b6b-df64-4e93-8b10-51a0e97f9441 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:44.748550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a48b6b-df64-4e93-8b10-51a0e97f9441 admin admin] PATH_INFO: `/v3/projects/4a78c33fc802413da0d7b3de9948374f/users/9439cabcc705477683d5956436197d65/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:44.768780 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a48b6b-df64-4e93-8b10-51a0e97f9441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a78c33fc802413da0d7b3de9948374f, user_id=9439cabcc705477683d5956436197d65, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:44.771482 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a48b6b-df64-4e93-8b10-51a0e97f9441 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:44.774763 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:44.789849 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1289/2570] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:44 2022] PUT /identity/v3/projects/4a78c33fc802413da0d7b3de9948374f/users/9439cabcc705477683d5956436197d65/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:44.794296 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35296c-21f7-44af-aebb-b64173585c52 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:44.808018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35296c-21f7-44af-aebb-b64173585c52 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:44.872242 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f2302-7cb2-4ca6-8e4f-9c283db66bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:44.872919 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f2302-7cb2-4ca6-8e4f-9c283db66bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:44.873627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f2302-7cb2-4ca6-8e4f-9c283db66bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:44.874135 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f2302-7cb2-4ca6-8e4f-9c283db66bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:44.874618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f2302-7cb2-4ca6-8e4f-9c283db66bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:44.876767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f2302-7cb2-4ca6-8e4f-9c283db66bb6 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:44.889207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1282/2571] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:44.893386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a48b6b-df64-4e93-8b10-51a0e97f9441 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:44.906185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a48b6b-df64-4e93-8b10-51a0e97f9441 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:44.920389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7301642-56b1-4e55-9e05-0a5850032444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:44.921292 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7301642-56b1-4e55-9e05-0a5850032444 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:44.921574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7301642-56b1-4e55-9e05-0a5850032444 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:44.921848 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7301642-56b1-4e55-9e05-0a5850032444 admin admin] PATH_INFO: `/v3/projects/4a78c33fc802413da0d7b3de9948374f/users/9439cabcc705477683d5956436197d65/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:44.923949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7301642-56b1-4e55-9e05-0a5850032444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a78c33fc802413da0d7b3de9948374f, user_id=9439cabcc705477683d5956436197d65, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:44.926837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7301642-56b1-4e55-9e05-0a5850032444 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:44.938250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1290/2572] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:44 2022] PUT /identity/v3/projects/4a78c33fc802413da0d7b3de9948374f/users/9439cabcc705477683d5956436197d65/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:45.879273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f76c2d-5106-4246-b036-94e34c7d54fa None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:45.879818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f76c2d-5106-4246-b036-94e34c7d54fa None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:45.879818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f76c2d-5106-4246-b036-94e34c7d54fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:45.911553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-74f76c2d-5106-4246-b036-94e34c7d54fa None None] MFA Rules not processed for user `9439cabcc705477683d5956436197d65`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:45.912835 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-74f76c2d-5106-4246-b036-94e34c7d54fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:45.976570 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:46.006748 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1283/2573] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:45 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:47.098379 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7301642-56b1-4e55-9e05-0a5850032444 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:47.170746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7301642-56b1-4e55-9e05-0a5850032444 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:47.177775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:47.178652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:47.178972 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:47.179254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] PATH_INFO: `/v3/users/7cdee476b12247178137555c3c555e57` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:47.180479 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cdee476b12247178137555c3c555e57)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:47.182411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:47.235142 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cdee476b12247178137555c3c555e57'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.244196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cdee476b12247178137555c3c555e57'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.253636 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cdee476b12247178137555c3c555e57'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.254382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cdee476b12247178137555c3c555e57'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.255632 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cdee476b12247178137555c3c555e57'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.280420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1291/2574] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:47 2022] DELETE /identity/v3/users/7cdee476b12247178137555c3c555e57 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:47.510900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f76c2d-5106-4246-b036-94e34c7d54fa None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:47.530259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f76c2d-5106-4246-b036-94e34c7d54fa None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:47.589406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:47.590361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:47.590582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:47.590843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] PATH_INFO: `/v3/projects/09301e0029c04454862ab0b7634458ab` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:47.592241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] RBAC: Authorizing `identity:delete_project(project_id=09301e0029c04454862ab0b7634458ab)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:47.594225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:47.642715 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09301e0029c04454862ab0b7634458ab', 'request_id': 'req-87c86f94-d2aa-4d93-a56c-2f05894072df'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.648805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09301e0029c04454862ab0b7634458ab', 'request_id': 'req-87c86f94-d2aa-4d93-a56c-2f05894072df'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.656837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09301e0029c04454862ab0b7634458ab', 'request_id': 'req-87c86f94-d2aa-4d93-a56c-2f05894072df'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.668386 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] The token cache is being invalidate because project 09301e0029c04454862ab0b7634458ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:47.668668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.669306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.670892 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1284/2575] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:47 2022] DELETE /identity/v3/projects/09301e0029c04454862ab0b7634458ab => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:47.674774 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:47.676222 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:47.690107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1759ea-9819-4760-95a6-5140b6f35a43 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:47.745880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1b76d-2cea-4b9b-82d5-71a2305d18be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:47.747365 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1b76d-2cea-4b9b-82d5-71a2305d18be admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:47.747668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1b76d-2cea-4b9b-82d5-71a2305d18be admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:47.747944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1b76d-2cea-4b9b-82d5-71a2305d18be admin admin] PATH_INFO: `/v3/users/9439cabcc705477683d5956436197d65` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:47.749124 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1b76d-2cea-4b9b-82d5-71a2305d18be admin admin] RBAC: Authorizing `identity:delete_user(user_id=9439cabcc705477683d5956436197d65)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:47.751067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1b76d-2cea-4b9b-82d5-71a2305d18be admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:47.753342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:47.798658 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-82b1b76d-2cea-4b9b-82d5-71a2305d18be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9439cabcc705477683d5956436197d65'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.805991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-82b1b76d-2cea-4b9b-82d5-71a2305d18be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9439cabcc705477683d5956436197d65'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.812951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-82b1b76d-2cea-4b9b-82d5-71a2305d18be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9439cabcc705477683d5956436197d65'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.813567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-82b1b76d-2cea-4b9b-82d5-71a2305d18be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9439cabcc705477683d5956436197d65'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.814174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-82b1b76d-2cea-4b9b-82d5-71a2305d18be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9439cabcc705477683d5956436197d65'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:47.830767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1292/2576] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:47 2022] DELETE /identity/v3/users/9439cabcc705477683d5956436197d65 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:48.009068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:48.010585 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:48.023991 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c86f94-d2aa-4d93-a56c-2f05894072df admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:48.081298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:48.082091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.082378 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.082656 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4 admin admin] PATH_INFO: `/v3/projects/4a78c33fc802413da0d7b3de9948374f` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.083882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a78c33fc802413da0d7b3de9948374f)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:48.085783 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:48.107767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:48.134078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a78c33fc802413da0d7b3de9948374f', 'request_id': 'req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:48.140606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a78c33fc802413da0d7b3de9948374f', 'request_id': 'req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:48.148116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a78c33fc802413da0d7b3de9948374f', 'request_id': 'req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:48.159403 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4 admin admin] The token cache is being invalidate because project 4a78c33fc802413da0d7b3de9948374f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:48.159692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:48.160330 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4c8d29e1-6f18-466f-9a1e-f7eb586b8dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:48.161956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1285/2577] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:48 2022] DELETE /identity/v3/projects/4a78c33fc802413da0d7b3de9948374f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:48.166912 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d9e49-ee11-4f83-a83c-14e30a427d21 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.167406 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d9e49-ee11-4f83-a83c-14e30a427d21 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.167511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d9e49-ee11-4f83-a83c-14e30a427d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.185872 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-579d9e49-ee11-4f83-a83c-14e30a427d21 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:48.187099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-579d9e49-ee11-4f83-a83c-14e30a427d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:48.246683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1293/2578] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:48.255524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e43690-3380-4a6c-99bb-5fd46e56ac31 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.255817 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e43690-3380-4a6c-99bb-5fd46e56ac31 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.256096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e43690-3380-4a6c-99bb-5fd46e56ac31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.275067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-30e43690-3380-4a6c-99bb-5fd46e56ac31 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:48.276255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-30e43690-3380-4a6c-99bb-5fd46e56ac31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:48.284856 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1286/2579] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:48.288942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d9e49-ee11-4f83-a83c-14e30a427d21 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:48.298530 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d9e49-ee11-4f83-a83c-14e30a427d21 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:48.303109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146b4d9-87f3-4880-8026-73fca50fe580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:48.303748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146b4d9-87f3-4880-8026-73fca50fe580 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.304029 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146b4d9-87f3-4880-8026-73fca50fe580 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.304301 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146b4d9-87f3-4880-8026-73fca50fe580 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.305015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146b4d9-87f3-4880-8026-73fca50fe580 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:01:48.305317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146b4d9-87f3-4880-8026-73fca50fe580 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:48.306910 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146b4d9-87f3-4880-8026-73fca50fe580 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:48.318297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1294/2580] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:01:48.322335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e43690-3380-4a6c-99bb-5fd46e56ac31 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:48.332587 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e43690-3380-4a6c-99bb-5fd46e56ac31 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:48.337401 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f608c0-dbfd-458a-b17e-77d6f9d69df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:48.338056 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f608c0-dbfd-458a-b17e-77d6f9d69df1 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.338339 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f608c0-dbfd-458a-b17e-77d6f9d69df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.338612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f608c0-dbfd-458a-b17e-77d6f9d69df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.339336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f608c0-dbfd-458a-b17e-77d6f9d69df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:48.341960 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f608c0-dbfd-458a-b17e-77d6f9d69df1 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:48.367629 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1287/2581] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:48 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:48.371935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146b4d9-87f3-4880-8026-73fca50fe580 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:48.382129 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146b4d9-87f3-4880-8026-73fca50fe580 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:48.463146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21764a7a-7505-4cbc-aeae-bb564af2bd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:48.463830 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21764a7a-7505-4cbc-aeae-bb564af2bd69 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.464121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21764a7a-7505-4cbc-aeae-bb564af2bd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.464408 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21764a7a-7505-4cbc-aeae-bb564af2bd69 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.465498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21764a7a-7505-4cbc-aeae-bb564af2bd69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:48.467577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21764a7a-7505-4cbc-aeae-bb564af2bd69 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:48.486817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:48.497898 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1295/2582] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:48 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:01:48.499207 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f608c0-dbfd-458a-b17e-77d6f9d69df1 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:48.509849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f608c0-dbfd-458a-b17e-77d6f9d69df1 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:48.514883 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb6ea3-1927-438f-b113-2e07dd5a09c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:48.515555 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb6ea3-1927-438f-b113-2e07dd5a09c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.515863 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb6ea3-1927-438f-b113-2e07dd5a09c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.516138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb6ea3-1927-438f-b113-2e07dd5a09c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.516922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb6ea3-1927-438f-b113-2e07dd5a09c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:48.518553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb6ea3-1927-438f-b113-2e07dd5a09c1 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:48.530058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1288/2583] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:48.534046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21764a7a-7505-4cbc-aeae-bb564af2bd69 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:48.545044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21764a7a-7505-4cbc-aeae-bb564af2bd69 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:48.549750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38836a0-3e3f-45b4-a3b3-8fcf5e1171db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:48.550388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38836a0-3e3f-45b4-a3b3-8fcf5e1171db admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.550673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38836a0-3e3f-45b4-a3b3-8fcf5e1171db admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.550990 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38836a0-3e3f-45b4-a3b3-8fcf5e1171db admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.551718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38836a0-3e3f-45b4-a3b3-8fcf5e1171db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:48.553309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38836a0-3e3f-45b4-a3b3-8fcf5e1171db admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:48.563451 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1296/2584] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:48.567581 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb6ea3-1927-438f-b113-2e07dd5a09c1 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:48.577714 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb6ea3-1927-438f-b113-2e07dd5a09c1 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:48.582391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867e585-6b3d-4120-bfc1-3698a20bf3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:48.583235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867e585-6b3d-4120-bfc1-3698a20bf3b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.583523 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867e585-6b3d-4120-bfc1-3698a20bf3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.583803 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867e585-6b3d-4120-bfc1-3698a20bf3b5 admin admin] PATH_INFO: `/v3/projects/12a6691b78604466baf066a1d984d30a/users/32746543b1234b76a209f13e1db97a07/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.598624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867e585-6b3d-4120-bfc1-3698a20bf3b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a6691b78604466baf066a1d984d30a, user_id=32746543b1234b76a209f13e1db97a07, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:48.601301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867e585-6b3d-4120-bfc1-3698a20bf3b5 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:48.614543 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1289/2585] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:48 2022] PUT /identity/v3/projects/12a6691b78604466baf066a1d984d30a/users/32746543b1234b76a209f13e1db97a07/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:48.624674 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a1b1b-230c-456e-aba1-3da7fa6c2cf7 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.624974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a1b1b-230c-456e-aba1-3da7fa6c2cf7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.625262 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a1b1b-230c-456e-aba1-3da7fa6c2cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.651450 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ae4a1b1b-230c-456e-aba1-3da7fa6c2cf7 None None] MFA Rules not processed for user `32746543b1234b76a209f13e1db97a07`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:48.653230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ae4a1b1b-230c-456e-aba1-3da7fa6c2cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:48.718232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1297/2586] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:48.722639 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867e585-6b3d-4120-bfc1-3698a20bf3b5 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:48.785065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867e585-6b3d-4120-bfc1-3698a20bf3b5 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:48.789733 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300cfe4-c6c9-405a-baf7-1063b2668e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:48.790389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300cfe4-c6c9-405a-baf7-1063b2668e47 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.790680 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300cfe4-c6c9-405a-baf7-1063b2668e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.790999 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300cfe4-c6c9-405a-baf7-1063b2668e47 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.791685 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300cfe4-c6c9-405a-baf7-1063b2668e47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:48.794299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300cfe4-c6c9-405a-baf7-1063b2668e47 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:48.823360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1290/2587] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:48 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:48.824604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a1b1b-230c-456e-aba1-3da7fa6c2cf7 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:48.835314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a1b1b-230c-456e-aba1-3da7fa6c2cf7 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:48.891755 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879614d-2345-4464-ad36-0ccedb40bb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:48.892416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879614d-2345-4464-ad36-0ccedb40bb61 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.892696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879614d-2345-4464-ad36-0ccedb40bb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.892967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879614d-2345-4464-ad36-0ccedb40bb61 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.893660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879614d-2345-4464-ad36-0ccedb40bb61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:48.895594 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879614d-2345-4464-ad36-0ccedb40bb61 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:48.915499 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1298/2588] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:48 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:01:48.920072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300cfe4-c6c9-405a-baf7-1063b2668e47 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:48.930178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300cfe4-c6c9-405a-baf7-1063b2668e47 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:48.934972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d754a8-c03d-4539-b7ba-d3e143f92f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:48.935618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d754a8-c03d-4539-b7ba-d3e143f92f53 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.935898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d754a8-c03d-4539-b7ba-d3e143f92f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.936171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d754a8-c03d-4539-b7ba-d3e143f92f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.936883 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d754a8-c03d-4539-b7ba-d3e143f92f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:48.938450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d754a8-c03d-4539-b7ba-d3e143f92f53 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:48.948777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1291/2589] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:48.953500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879614d-2345-4464-ad36-0ccedb40bb61 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:48.963387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879614d-2345-4464-ad36-0ccedb40bb61 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:48.969067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edea8d-5279-4c11-a27b-bcb495035024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:48.969716 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edea8d-5279-4c11-a27b-bcb495035024 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:48.970009 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edea8d-5279-4c11-a27b-bcb495035024 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:48.970289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edea8d-5279-4c11-a27b-bcb495035024 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:48.971095 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20edea8d-5279-4c11-a27b-bcb495035024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:48.972646 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20edea8d-5279-4c11-a27b-bcb495035024 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:48.983088 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1299/2590] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:48.987038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d754a8-c03d-4539-b7ba-d3e143f92f53 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:48.996881 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d754a8-c03d-4539-b7ba-d3e143f92f53 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:49.001469 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17563eb-9a8b-4321-841b-0432cd269d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:49.002328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17563eb-9a8b-4321-841b-0432cd269d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:49.002643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17563eb-9a8b-4321-841b-0432cd269d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:49.003003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17563eb-9a8b-4321-841b-0432cd269d4f admin admin] PATH_INFO: `/v3/projects/e0909028923a46099025925b502baf08/users/fa7d091ff55b495bbb2e0ff900c3a32b/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:49.018359 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17563eb-9a8b-4321-841b-0432cd269d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0909028923a46099025925b502baf08, user_id=fa7d091ff55b495bbb2e0ff900c3a32b, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:49.021246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17563eb-9a8b-4321-841b-0432cd269d4f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:49.023599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:49.036816 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1292/2591] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:48 2022] PUT /identity/v3/projects/e0909028923a46099025925b502baf08/users/fa7d091ff55b495bbb2e0ff900c3a32b/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:49.047349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4c3ef-4990-44c1-a9c3-70cc6ff53a51 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:49.047643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4c3ef-4990-44c1-a9c3-70cc6ff53a51 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:49.047922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4c3ef-4990-44c1-a9c3-70cc6ff53a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:49.074989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-3fa4c3ef-4990-44c1-a9c3-70cc6ff53a51 None None] MFA Rules not processed for user `fa7d091ff55b495bbb2e0ff900c3a32b`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:49.076202 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3fa4c3ef-4990-44c1-a9c3-70cc6ff53a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:49.123438 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:49.147994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1300/2592] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:49.153248 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17563eb-9a8b-4321-841b-0432cd269d4f admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:49.219867 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17563eb-9a8b-4321-841b-0432cd269d4f admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:49.224613 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabb6f2-073c-4ab6-b449-78a8a9f3634c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:49.225268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabb6f2-073c-4ab6-b449-78a8a9f3634c admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:49.225548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabb6f2-073c-4ab6-b449-78a8a9f3634c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:49.225821 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabb6f2-073c-4ab6-b449-78a8a9f3634c admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:49.226500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabb6f2-073c-4ab6-b449-78a8a9f3634c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:49.228409 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabb6f2-073c-4ab6-b449-78a8a9f3634c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:49.259571 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1293/2593] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:49 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:49.260720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4c3ef-4990-44c1-a9c3-70cc6ff53a51 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:49.271762 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4c3ef-4990-44c1-a9c3-70cc6ff53a51 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:49.329747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acdeba-644c-4d59-8483-5baa025ef905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:49.330414 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acdeba-644c-4d59-8483-5baa025ef905 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:49.330696 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acdeba-644c-4d59-8483-5baa025ef905 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:49.331015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acdeba-644c-4d59-8483-5baa025ef905 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:49.331698 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acdeba-644c-4d59-8483-5baa025ef905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:49.333603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acdeba-644c-4d59-8483-5baa025ef905 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:49.353329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1301/2594] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:49 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:49.357984 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabb6f2-073c-4ab6-b449-78a8a9f3634c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:49.368668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabb6f2-073c-4ab6-b449-78a8a9f3634c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:49.373351 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0232f3-aa0c-4b39-8b05-535ee32a9d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:49.374058 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0232f3-aa0c-4b39-8b05-535ee32a9d6e admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:49.374293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0232f3-aa0c-4b39-8b05-535ee32a9d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:49.374574 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0232f3-aa0c-4b39-8b05-535ee32a9d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:49.375364 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0232f3-aa0c-4b39-8b05-535ee32a9d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:49.376949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0232f3-aa0c-4b39-8b05-535ee32a9d6e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:49.388835 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1294/2595] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:49.392903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acdeba-644c-4d59-8483-5baa025ef905 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:49.403881 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acdeba-644c-4d59-8483-5baa025ef905 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:49.408836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36956be-dbfd-4d31-accb-fd67a89369ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:49.410281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36956be-dbfd-4d31-accb-fd67a89369ed admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:49.410779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36956be-dbfd-4d31-accb-fd67a89369ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:49.411330 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36956be-dbfd-4d31-accb-fd67a89369ed admin admin] PATH_INFO: `/v3/projects/29123620661b426ea1bfdb01386d496e/users/acc2154969514861bfce6ea03f34a604/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:49.436649 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36956be-dbfd-4d31-accb-fd67a89369ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=29123620661b426ea1bfdb01386d496e, user_id=acc2154969514861bfce6ea03f34a604, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:49.439631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36956be-dbfd-4d31-accb-fd67a89369ed admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:49.452343 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1302/2596] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:49 2022] PUT /identity/v3/projects/29123620661b426ea1bfdb01386d496e/users/acc2154969514861bfce6ea03f34a604/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:49.456301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0232f3-aa0c-4b39-8b05-535ee32a9d6e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:49.466917 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0232f3-aa0c-4b39-8b05-535ee32a9d6e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:49.525778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd07a2-525e-47ac-a65f-59f086ebd87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:49.526444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd07a2-525e-47ac-a65f-59f086ebd87e admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:49.526726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd07a2-525e-47ac-a65f-59f086ebd87e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:49.527655 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd07a2-525e-47ac-a65f-59f086ebd87e admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:49.528649 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd07a2-525e-47ac-a65f-59f086ebd87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:49.530217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd07a2-525e-47ac-a65f-59f086ebd87e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:49.540112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1295/2597] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:49.544795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36956be-dbfd-4d31-accb-fd67a89369ed admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:49.555165 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36956be-dbfd-4d31-accb-fd67a89369ed admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:49.559892 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0663f7e-dc50-4ab9-920f-42fad27ba924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:49.560780 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0663f7e-dc50-4ab9-920f-42fad27ba924 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:49.561065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0663f7e-dc50-4ab9-920f-42fad27ba924 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:49.561340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0663f7e-dc50-4ab9-920f-42fad27ba924 admin admin] PATH_INFO: `/v3/projects/29123620661b426ea1bfdb01386d496e/users/acc2154969514861bfce6ea03f34a604/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:49.563348 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0663f7e-dc50-4ab9-920f-42fad27ba924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29123620661b426ea1bfdb01386d496e, user_id=acc2154969514861bfce6ea03f34a604, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:49.565966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0663f7e-dc50-4ab9-920f-42fad27ba924 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:49.587153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1303/2598] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:49 2022] PUT /identity/v3/projects/29123620661b426ea1bfdb01386d496e/users/acc2154969514861bfce6ea03f34a604/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:49.588775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8426cc-fb53-4f64-9256-71b6f1f3d0da None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:49.589071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8426cc-fb53-4f64-9256-71b6f1f3d0da None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:49.589354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8426cc-fb53-4f64-9256-71b6f1f3d0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:49.619456 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6d8426cc-fb53-4f64-9256-71b6f1f3d0da None None] MFA Rules not processed for user `acc2154969514861bfce6ea03f34a604`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:49.620829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6d8426cc-fb53-4f64-9256-71b6f1f3d0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:49.678621 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:49.698327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1296/2599] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:49.710337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0663f7e-dc50-4ab9-920f-42fad27ba924 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:49.712119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c0663f7e-dc50-4ab9-920f-42fad27ba924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:49.773108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0663f7e-dc50-4ab9-920f-42fad27ba924 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:49.776808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303e3ef-88c8-4712-92a8-ed1cc0eee14b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:49.777544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303e3ef-88c8-4712-92a8-ed1cc0eee14b service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:49.777825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303e3ef-88c8-4712-92a8-ed1cc0eee14b service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:49.778099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303e3ef-88c8-4712-92a8-ed1cc0eee14b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:49.787133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303e3ef-88c8-4712-92a8-ed1cc0eee14b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:49.788551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303e3ef-88c8-4712-92a8-ed1cc0eee14b service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:49.798425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1304/2600] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:49.840479 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8426cc-fb53-4f64-9256-71b6f1f3d0da None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:49.841850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6d8426cc-fb53-4f64-9256-71b6f1f3d0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:49.853842 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8426cc-fb53-4f64-9256-71b6f1f3d0da None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:49.858170 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e62c6f-3caa-48c7-9d7d-856411875049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:49.858790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e62c6f-3caa-48c7-9d7d-856411875049 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:49.859108 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e62c6f-3caa-48c7-9d7d-856411875049 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:49.859381 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e62c6f-3caa-48c7-9d7d-856411875049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:49.916121 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e62c6f-3caa-48c7-9d7d-856411875049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:49.917523 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e62c6f-3caa-48c7-9d7d-856411875049 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:49.927218 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1297/2601] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:52.818173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303e3ef-88c8-4712-92a8-ed1cc0eee14b service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:52.864363 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:52.885562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303e3ef-88c8-4712-92a8-ed1cc0eee14b service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:52.890327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:52.891240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:52.891522 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:52.891797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] PATH_INFO: `/v3/users/32746543b1234b76a209f13e1db97a07` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:52.893004 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32746543b1234b76a209f13e1db97a07)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:52.895015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:52.943006 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32746543b1234b76a209f13e1db97a07'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:52.950338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32746543b1234b76a209f13e1db97a07'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:52.957316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32746543b1234b76a209f13e1db97a07'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:52.957949 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32746543b1234b76a209f13e1db97a07'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:52.958560 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32746543b1234b76a209f13e1db97a07'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:52.976019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1305/2602] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:52 2022] DELETE /identity/v3/users/32746543b1234b76a209f13e1db97a07 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:52.981617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1ce22-6bab-478c-b08f-4bc76404ef24 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:52.982090 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1ce22-6bab-478c-b08f-4bc76404ef24 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:52.982203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1ce22-6bab-478c-b08f-4bc76404ef24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:52.999553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-68c1ce22-6bab-478c-b08f-4bc76404ef24 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:53.000756 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-68c1ce22-6bab-478c-b08f-4bc76404ef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:53.071389 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1298/2603] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:53.086153 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:53.087856 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:53.152751 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb3c0e-74cd-45e6-a815-8164bcc12242 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:53.156797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdc1dc-d21b-42e6-a5b9-a92e8e9b3de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:53.157522 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdc1dc-d21b-42e6-a5b9-a92e8e9b3de1 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:53.157882 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdc1dc-d21b-42e6-a5b9-a92e8e9b3de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:53.158234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdc1dc-d21b-42e6-a5b9-a92e8e9b3de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:53.168492 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdc1dc-d21b-42e6-a5b9-a92e8e9b3de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:53.170297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdc1dc-d21b-42e6-a5b9-a92e8e9b3de1 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:53.180327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1306/2604] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:01:53.464265 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1ce22-6bab-478c-b08f-4bc76404ef24 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:53.466467 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-68c1ce22-6bab-478c-b08f-4bc76404ef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:53.485789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1ce22-6bab-478c-b08f-4bc76404ef24 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:53.491173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:53.492167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:53.492624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:53.492945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] PATH_INFO: `/v3/projects/12a6691b78604466baf066a1d984d30a` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:53.494345 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a6691b78604466baf066a1d984d30a)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:53.496310 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:53.548869 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a6691b78604466baf066a1d984d30a', 'request_id': 'req-b2f50721-a208-4ad9-a1dd-4a68142f0440'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:53.555674 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a6691b78604466baf066a1d984d30a', 'request_id': 'req-b2f50721-a208-4ad9-a1dd-4a68142f0440'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:53.573235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a6691b78604466baf066a1d984d30a', 'request_id': 'req-b2f50721-a208-4ad9-a1dd-4a68142f0440'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:53.579140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] The token cache is being invalidate because project 12a6691b78604466baf066a1d984d30a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:53.579436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:53.580172 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:53.581862 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1299/2605] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:53 2022] DELETE /identity/v3/projects/12a6691b78604466baf066a1d984d30a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:53.587242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdc1dc-d21b-42e6-a5b9-a92e8e9b3de1 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:53.597818 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdc1dc-d21b-42e6-a5b9-a92e8e9b3de1 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:53.665358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:53.666232 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:53.666517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:53.666793 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] PATH_INFO: `/v3/users/fa7d091ff55b495bbb2e0ff900c3a32b` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:53.668187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa7d091ff55b495bbb2e0ff900c3a32b)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:53.670187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:53.672580 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:53.719233 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa7d091ff55b495bbb2e0ff900c3a32b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:53.726502 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa7d091ff55b495bbb2e0ff900c3a32b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:53.733319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa7d091ff55b495bbb2e0ff900c3a32b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:53.733941 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa7d091ff55b495bbb2e0ff900c3a32b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:53.734551 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa7d091ff55b495bbb2e0ff900c3a32b'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:53.752416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1307/2606] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:53 2022] DELETE /identity/v3/users/fa7d091ff55b495bbb2e0ff900c3a32b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:54.031830 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:54.033587 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:54.049893 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f50721-a208-4ad9-a1dd-4a68142f0440 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:54.061267 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:54.113151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:54.114048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:54.114337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:54.114618 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] PATH_INFO: `/v3/projects/e0909028923a46099025925b502baf08` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:54.115869 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0909028923a46099025925b502baf08)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:54.117773 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:54.170461 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0909028923a46099025925b502baf08', 'request_id': 'req-007ab75a-d197-46da-aad3-847ed51a8399'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.176560 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0909028923a46099025925b502baf08', 'request_id': 'req-007ab75a-d197-46da-aad3-847ed51a8399'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.184551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0909028923a46099025925b502baf08', 'request_id': 'req-007ab75a-d197-46da-aad3-847ed51a8399'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.194731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] The token cache is being invalidate because project e0909028923a46099025925b502baf08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:54.195053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.195658 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.197188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1300/2607] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:54 2022] DELETE /identity/v3/projects/e0909028923a46099025925b502baf08 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:54.201390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:54.202902 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:54.216241 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02007183-e60b-443b-bcdf-9b4c3e983524 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:54.276222 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96fb16-3510-4195-9384-5589af961fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:54.277300 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96fb16-3510-4195-9384-5589af961fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:54.277451 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96fb16-3510-4195-9384-5589af961fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:54.277772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96fb16-3510-4195-9384-5589af961fe3 admin admin] PATH_INFO: `/v3/users/acc2154969514861bfce6ea03f34a604` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:54.279006 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96fb16-3510-4195-9384-5589af961fe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acc2154969514861bfce6ea03f34a604)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:54.280919 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96fb16-3510-4195-9384-5589af961fe3 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:54.332877 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2b96fb16-3510-4195-9384-5589af961fe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acc2154969514861bfce6ea03f34a604'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.339214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2b96fb16-3510-4195-9384-5589af961fe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acc2154969514861bfce6ea03f34a604'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.346114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2b96fb16-3510-4195-9384-5589af961fe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acc2154969514861bfce6ea03f34a604'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.346731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2b96fb16-3510-4195-9384-5589af961fe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acc2154969514861bfce6ea03f34a604'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.347365 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2b96fb16-3510-4195-9384-5589af961fe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acc2154969514861bfce6ea03f34a604'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.361070 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:54.366076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1308/2608] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:01:54 2022] DELETE /identity/v3/users/acc2154969514861bfce6ea03f34a604 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:54.652952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:54.655624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:54.675674 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ab75a-d197-46da-aad3-847ed51a8399 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:54.735985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:54.736844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:54.737138 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:54.737402 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7 admin admin] PATH_INFO: `/v3/projects/29123620661b426ea1bfdb01386d496e` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:54.738577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29123620661b426ea1bfdb01386d496e)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:54.740517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:54.785860 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:54.790104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29123620661b426ea1bfdb01386d496e', 'request_id': 'req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.797804 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29123620661b426ea1bfdb01386d496e', 'request_id': 'req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.805095 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29123620661b426ea1bfdb01386d496e', 'request_id': 'req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.816017 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7 admin admin] The token cache is being invalidate because project 29123620661b426ea1bfdb01386d496e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:01:54.816245 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.816855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-4a4d56eb-c6b8-411c-9e57-f6b4d3eceed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:01:54.818401 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1301/2609] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:01:54 2022] DELETE /identity/v3/projects/29123620661b426ea1bfdb01386d496e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:54.824617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194f8f8-903e-40be-80e9-52512c92c558 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:54.824905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194f8f8-903e-40be-80e9-52512c92c558 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:54.825208 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194f8f8-903e-40be-80e9-52512c92c558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:54.844655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-3194f8f8-903e-40be-80e9-52512c92c558 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:54.845890 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3194f8f8-903e-40be-80e9-52512c92c558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:54.908230 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1309/2610] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:54.916957 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3c1ac-fd1a-4b5b-a447-740177dd7866 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:54.917253 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3c1ac-fd1a-4b5b-a447-740177dd7866 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:54.917598 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3c1ac-fd1a-4b5b-a447-740177dd7866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:54.935823 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-b4a3c1ac-fd1a-4b5b-a447-740177dd7866 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:54.937026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b4a3c1ac-fd1a-4b5b-a447-740177dd7866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:54.944958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1302/2611] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:01:54.950007 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194f8f8-903e-40be-80e9-52512c92c558 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:54.959619 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194f8f8-903e-40be-80e9-52512c92c558 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:54.964280 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc477c2-75e9-4263-b94e-6d453304eed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:54.965728 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc477c2-75e9-4263-b94e-6d453304eed6 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:54.966022 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc477c2-75e9-4263-b94e-6d453304eed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:54.966303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc477c2-75e9-4263-b94e-6d453304eed6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:54.967141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc477c2-75e9-4263-b94e-6d453304eed6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:01:54.967452 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc477c2-75e9-4263-b94e-6d453304eed6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:54.969046 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc477c2-75e9-4263-b94e-6d453304eed6 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:54.980524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1310/2612] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:01:54.984972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3c1ac-fd1a-4b5b-a447-740177dd7866 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:54.994011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3c1ac-fd1a-4b5b-a447-740177dd7866 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:54.998844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238aa31b-417d-4e17-b011-b71ca8011d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:54.999511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238aa31b-417d-4e17-b011-b71ca8011d8f admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:54.999793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238aa31b-417d-4e17-b011-b71ca8011d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.000091 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238aa31b-417d-4e17-b011-b71ca8011d8f admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:55.000775 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238aa31b-417d-4e17-b011-b71ca8011d8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:55.002644 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238aa31b-417d-4e17-b011-b71ca8011d8f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:55.032071 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1303/2613] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:54 2022] POST /identity/v3/projects => generated 385 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:55.033511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc477c2-75e9-4263-b94e-6d453304eed6 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:55.045678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc477c2-75e9-4263-b94e-6d453304eed6 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:55.101282 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5e1a8-4e27-4923-8005-0097f52a46e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:55.101859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5e1a8-4e27-4923-8005-0097f52a46e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:55.102141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5e1a8-4e27-4923-8005-0097f52a46e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.102414 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5e1a8-4e27-4923-8005-0097f52a46e2 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:55.103149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5e1a8-4e27-4923-8005-0097f52a46e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:55.105026 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5e1a8-4e27-4923-8005-0097f52a46e2 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:55.124320 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1311/2614] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:55 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:55.128464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238aa31b-417d-4e17-b011-b71ca8011d8f admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:55.139581 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238aa31b-417d-4e17-b011-b71ca8011d8f admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:55.144161 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cf46f-5408-4c67-93e1-bd31b8da581f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:55.144807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cf46f-5408-4c67-93e1-bd31b8da581f admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:55.145087 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cf46f-5408-4c67-93e1-bd31b8da581f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.145363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cf46f-5408-4c67-93e1-bd31b8da581f admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:55.146078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5cf46f-5408-4c67-93e1-bd31b8da581f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:55.147680 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5cf46f-5408-4c67-93e1-bd31b8da581f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:55.160344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1304/2615] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:55.164579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5e1a8-4e27-4923-8005-0097f52a46e2 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:55.175603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5e1a8-4e27-4923-8005-0097f52a46e2 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:55.180867 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef7433-0b41-4592-9993-04ee02171a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:55.181547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef7433-0b41-4592-9993-04ee02171a01 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:55.181836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef7433-0b41-4592-9993-04ee02171a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.182115 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef7433-0b41-4592-9993-04ee02171a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:55.182893 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef7433-0b41-4592-9993-04ee02171a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:55.184485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef7433-0b41-4592-9993-04ee02171a01 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:55.195119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1312/2616] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:01:55.199854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cf46f-5408-4c67-93e1-bd31b8da581f admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:55.209700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cf46f-5408-4c67-93e1-bd31b8da581f admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:55.214395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be15192-194c-420c-8a44-008341079d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:55.215906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be15192-194c-420c-8a44-008341079d20 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:55.216171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be15192-194c-420c-8a44-008341079d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.216438 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be15192-194c-420c-8a44-008341079d20 admin admin] PATH_INFO: `/v3/projects/146f6bf548cf4a2183eb0cd3b17a23dc/users/7c2440dec9ea4127b0184d5551725b32/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:55.231019 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be15192-194c-420c-8a44-008341079d20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=146f6bf548cf4a2183eb0cd3b17a23dc, user_id=7c2440dec9ea4127b0184d5551725b32, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:55.233680 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be15192-194c-420c-8a44-008341079d20 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:55.247951 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1305/2617] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:55 2022] PUT /identity/v3/projects/146f6bf548cf4a2183eb0cd3b17a23dc/users/7c2440dec9ea4127b0184d5551725b32/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:55.260317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84644a3f-fe3c-424e-b425-6b9afe7265e0 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:55.260800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84644a3f-fe3c-424e-b425-6b9afe7265e0 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.260911 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84644a3f-fe3c-424e-b425-6b9afe7265e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:55.287722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-84644a3f-fe3c-424e-b425-6b9afe7265e0 None None] MFA Rules not processed for user `7c2440dec9ea4127b0184d5551725b32`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:55.288933 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-84644a3f-fe3c-424e-b425-6b9afe7265e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:55.348517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:55.358313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1313/2618] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:55.362749 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be15192-194c-420c-8a44-008341079d20 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:55.423727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be15192-194c-420c-8a44-008341079d20 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:55.428451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb48c2b-7e96-4716-99ac-249420463fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:55.429104 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb48c2b-7e96-4716-99ac-249420463fce admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:55.429384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb48c2b-7e96-4716-99ac-249420463fce admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.429661 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb48c2b-7e96-4716-99ac-249420463fce admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:55.430340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb48c2b-7e96-4716-99ac-249420463fce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:55.432255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb48c2b-7e96-4716-99ac-249420463fce admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:55.458311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1306/2619] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:01:55 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:55.462352 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84644a3f-fe3c-424e-b425-6b9afe7265e0 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:55.472979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84644a3f-fe3c-424e-b425-6b9afe7265e0 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:55.532945 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24da44-04bc-4667-9068-4d03abe3fc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:55.533614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24da44-04bc-4667-9068-4d03abe3fc3b admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:55.533898 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24da44-04bc-4667-9068-4d03abe3fc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.534179 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24da44-04bc-4667-9068-4d03abe3fc3b admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:55.534893 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24da44-04bc-4667-9068-4d03abe3fc3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:55.536792 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24da44-04bc-4667-9068-4d03abe3fc3b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:55.556768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1314/2620] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:01:55 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:01:55.561253 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb48c2b-7e96-4716-99ac-249420463fce admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:55.571627 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb48c2b-7e96-4716-99ac-249420463fce admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:55.576870 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59b8da-ba98-44e9-ae4b-7b006e99b41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:55.577532 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59b8da-ba98-44e9-ae4b-7b006e99b41d admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:55.577819 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59b8da-ba98-44e9-ae4b-7b006e99b41d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.578100 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59b8da-ba98-44e9-ae4b-7b006e99b41d admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:55.578825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59b8da-ba98-44e9-ae4b-7b006e99b41d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:55.580446 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59b8da-ba98-44e9-ae4b-7b006e99b41d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:55.590789 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1307/2621] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:55.594928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24da44-04bc-4667-9068-4d03abe3fc3b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:55.604771 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24da44-04bc-4667-9068-4d03abe3fc3b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:55.609983 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5630aa2-1dae-4f86-8f82-dcbb1d95f2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:55.610925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5630aa2-1dae-4f86-8f82-dcbb1d95f2c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:55.611238 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5630aa2-1dae-4f86-8f82-dcbb1d95f2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.611500 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5630aa2-1dae-4f86-8f82-dcbb1d95f2c1 admin admin] PATH_INFO: `/v3/projects/1644373b4bf146a7a53c18de1722fb77/users/5d7b23c8bab04dca94d6b74b6fd63e25/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:55.626464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5630aa2-1dae-4f86-8f82-dcbb1d95f2c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1644373b4bf146a7a53c18de1722fb77, user_id=5d7b23c8bab04dca94d6b74b6fd63e25, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:55.629483 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5630aa2-1dae-4f86-8f82-dcbb1d95f2c1 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:55.642377 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1315/2622] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:55 2022] PUT /identity/v3/projects/1644373b4bf146a7a53c18de1722fb77/users/5d7b23c8bab04dca94d6b74b6fd63e25/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:01:55.646306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59b8da-ba98-44e9-ae4b-7b006e99b41d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:55.657956 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59b8da-ba98-44e9-ae4b-7b006e99b41d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:55.704466 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:01:55.723522 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c93b1-416e-4f11-84a7-d842f11649d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:55.724263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c93b1-416e-4f11-84a7-d842f11649d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:55.724617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c93b1-416e-4f11-84a7-d842f11649d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.724961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c93b1-416e-4f11-84a7-d842f11649d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:55.725875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c93b1-416e-4f11-84a7-d842f11649d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:55.727844 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c93b1-416e-4f11-84a7-d842f11649d7 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:55.738227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1308/2623] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:01:55.742297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5630aa2-1dae-4f86-8f82-dcbb1d95f2c1 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:55.753525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5630aa2-1dae-4f86-8f82-dcbb1d95f2c1 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:55.758398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b71f6-108f-4204-9a88-81ee5a885467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:55.759290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b71f6-108f-4204-9a88-81ee5a885467 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:55.759572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b71f6-108f-4204-9a88-81ee5a885467 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.759847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b71f6-108f-4204-9a88-81ee5a885467 admin admin] PATH_INFO: `/v3/projects/1644373b4bf146a7a53c18de1722fb77/users/5d7b23c8bab04dca94d6b74b6fd63e25/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:55.761824 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b71f6-108f-4204-9a88-81ee5a885467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1644373b4bf146a7a53c18de1722fb77, user_id=5d7b23c8bab04dca94d6b74b6fd63e25, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:55.764614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b71f6-108f-4204-9a88-81ee5a885467 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:55.774612 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1316/2624] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:01:55 2022] PUT /identity/v3/projects/1644373b4bf146a7a53c18de1722fb77/users/5d7b23c8bab04dca94d6b74b6fd63e25/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:01:55.784846 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d44f53-8c48-4392-825d-523465e90a46 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:55.785205 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d44f53-8c48-4392-825d-523465e90a46 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.785416 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d44f53-8c48-4392-825d-523465e90a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:55.814972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-55d44f53-8c48-4392-825d-523465e90a46 None None] MFA Rules not processed for user `5d7b23c8bab04dca94d6b74b6fd63e25`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:01:55.816335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-55d44f53-8c48-4392-825d-523465e90a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:55.886275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1309/2625] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:01:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:01:55.897437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b71f6-108f-4204-9a88-81ee5a885467 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:01:55.899208 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-4b1b71f6-108f-4204-9a88-81ee5a885467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:01:55.957016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b71f6-108f-4204-9a88-81ee5a885467 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:01:55.961306 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f158052-29dc-48b7-9e86-27e2b152799e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:01:55.962102 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f158052-29dc-48b7-9e86-27e2b152799e service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:01:55.962395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f158052-29dc-48b7-9e86-27e2b152799e service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:01:55.962677 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f158052-29dc-48b7-9e86-27e2b152799e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:01:56.021938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f158052-29dc-48b7-9e86-27e2b152799e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:01:56.023421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f158052-29dc-48b7-9e86-27e2b152799e service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:01:56.033089 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1317/2626] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:01:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:02:00.227009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d44f53-8c48-4392-825d-523465e90a46 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:00.228426 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-55d44f53-8c48-4392-825d-523465e90a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:00.241151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d44f53-8c48-4392-825d-523465e90a46 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:00.244661 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b012d-5872-4374-a276-92047f40e0db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:00.245293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b012d-5872-4374-a276-92047f40e0db service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:00.245603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b012d-5872-4374-a276-92047f40e0db service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:00.245898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b012d-5872-4374-a276-92047f40e0db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:00.255225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b012d-5872-4374-a276-92047f40e0db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:00.256988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b012d-5872-4374-a276-92047f40e0db service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:00.267166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1310/2627] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:02:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:02:05.143758 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f158052-29dc-48b7-9e86-27e2b152799e service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:05.148207 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:05.210325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f158052-29dc-48b7-9e86-27e2b152799e service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:05.214841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:05.216270 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:05.216554 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:05.216835 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] PATH_INFO: `/v3/users/7c2440dec9ea4127b0184d5551725b32` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:05.218035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2440dec9ea4127b0184d5551725b32)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:05.219992 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:05.267210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2440dec9ea4127b0184d5551725b32'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:05.274603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2440dec9ea4127b0184d5551725b32'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:05.283158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2440dec9ea4127b0184d5551725b32'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:05.283922 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2440dec9ea4127b0184d5551725b32'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:05.284763 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2440dec9ea4127b0184d5551725b32'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:05.303196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1318/2628] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:02:05 2022] DELETE /identity/v3/users/7c2440dec9ea4127b0184d5551725b32 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:05.307680 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f6239-3cf1-460d-aeb8-000c1fb66ad6 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:05.308150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f6239-3cf1-460d-aeb8-000c1fb66ad6 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:05.308255 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f6239-3cf1-460d-aeb8-000c1fb66ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:05.325898 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-cd1f6239-3cf1-460d-aeb8-000c1fb66ad6 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:05.327200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-cd1f6239-3cf1-460d-aeb8-000c1fb66ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:05.383882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1311/2629] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:02:05.394325 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:05.395944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:05.453753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c6e49-1272-448f-a845-7e9affff7374 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:05.457513 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c295f7-f864-4f99-8388-b66988260eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:05.458154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c295f7-f864-4f99-8388-b66988260eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:05.458442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c295f7-f864-4f99-8388-b66988260eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:05.458722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c295f7-f864-4f99-8388-b66988260eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:05.468548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c295f7-f864-4f99-8388-b66988260eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:05.469974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c295f7-f864-4f99-8388-b66988260eb4 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:05.479314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1319/2630] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:02:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:02:05.756811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f6239-3cf1-460d-aeb8-000c1fb66ad6 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:05.758414 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-cd1f6239-3cf1-460d-aeb8-000c1fb66ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:05.771979 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f6239-3cf1-460d-aeb8-000c1fb66ad6 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:05.777159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:05.778145 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:05.778431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:05.778771 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] PATH_INFO: `/v3/projects/146f6bf548cf4a2183eb0cd3b17a23dc` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:05.780709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=146f6bf548cf4a2183eb0cd3b17a23dc)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:05.782688 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:05.830573 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:05.833194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '146f6bf548cf4a2183eb0cd3b17a23dc', 'request_id': 'req-ad046ff4-43ce-44b1-ae31-4d17a91accc0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:05.840597 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '146f6bf548cf4a2183eb0cd3b17a23dc', 'request_id': 'req-ad046ff4-43ce-44b1-ae31-4d17a91accc0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:05.848869 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '146f6bf548cf4a2183eb0cd3b17a23dc', 'request_id': 'req-ad046ff4-43ce-44b1-ae31-4d17a91accc0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:05.860141 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] The token cache is being invalidate because project 146f6bf548cf4a2183eb0cd3b17a23dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:02:05.860431 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:05.861102 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:05.862667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1312/2631] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:02:05 2022] DELETE /identity/v3/projects/146f6bf548cf4a2183eb0cd3b17a23dc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:05.866854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c295f7-f864-4f99-8388-b66988260eb4 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:05.879064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c295f7-f864-4f99-8388-b66988260eb4 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:05.901326 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:05.942736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7c046-0a36-4244-8276-7f0c74ea7c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:05.943661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7c046-0a36-4244-8276-7f0c74ea7c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:05.943944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7c046-0a36-4244-8276-7f0c74ea7c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:05.944218 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7c046-0a36-4244-8276-7f0c74ea7c95 admin admin] PATH_INFO: `/v3/users/5d7b23c8bab04dca94d6b74b6fd63e25` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:05.945410 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7c046-0a36-4244-8276-7f0c74ea7c95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d7b23c8bab04dca94d6b74b6fd63e25)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:05.947379 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7c046-0a36-4244-8276-7f0c74ea7c95 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:05.993324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-07b7c046-0a36-4244-8276-7f0c74ea7c95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d7b23c8bab04dca94d6b74b6fd63e25'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:06.002006 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-07b7c046-0a36-4244-8276-7f0c74ea7c95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d7b23c8bab04dca94d6b74b6fd63e25'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:06.010072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-07b7c046-0a36-4244-8276-7f0c74ea7c95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d7b23c8bab04dca94d6b74b6fd63e25'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:06.010744 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-07b7c046-0a36-4244-8276-7f0c74ea7c95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d7b23c8bab04dca94d6b74b6fd63e25'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:06.011463 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-07b7c046-0a36-4244-8276-7f0c74ea7c95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d7b23c8bab04dca94d6b74b6fd63e25'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:06.029723 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1320/2632] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:02:05 2022] DELETE /identity/v3/users/5d7b23c8bab04dca94d6b74b6fd63e25 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:06.313453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:06.315030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:06.329335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad046ff4-43ce-44b1-ae31-4d17a91accc0 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:06.387399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095e592-df58-4084-85cb-388e6aa50bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:06.388350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095e592-df58-4084-85cb-388e6aa50bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:06.388635 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095e592-df58-4084-85cb-388e6aa50bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:06.388908 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095e592-df58-4084-85cb-388e6aa50bf3 admin admin] PATH_INFO: `/v3/projects/1644373b4bf146a7a53c18de1722fb77` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:06.390277 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095e592-df58-4084-85cb-388e6aa50bf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1644373b4bf146a7a53c18de1722fb77)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:06.392254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095e592-df58-4084-85cb-388e6aa50bf3 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:06.439152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8095e592-df58-4084-85cb-388e6aa50bf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1644373b4bf146a7a53c18de1722fb77', 'request_id': 'req-8095e592-df58-4084-85cb-388e6aa50bf3'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:06.445274 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8095e592-df58-4084-85cb-388e6aa50bf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1644373b4bf146a7a53c18de1722fb77', 'request_id': 'req-8095e592-df58-4084-85cb-388e6aa50bf3'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:06.452447 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8095e592-df58-4084-85cb-388e6aa50bf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1644373b4bf146a7a53c18de1722fb77', 'request_id': 'req-8095e592-df58-4084-85cb-388e6aa50bf3'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:06.462594 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8095e592-df58-4084-85cb-388e6aa50bf3 admin admin] The token cache is being invalidate because project 1644373b4bf146a7a53c18de1722fb77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:02:06.462903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8095e592-df58-4084-85cb-388e6aa50bf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:06.463524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8095e592-df58-4084-85cb-388e6aa50bf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:06.465093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1313/2633] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:02:06 2022] DELETE /identity/v3/projects/1644373b4bf146a7a53c18de1722fb77 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:06.471772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c3deb-7f1a-4c09-8e43-90dc1e7d28b9 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:06.472284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c3deb-7f1a-4c09-8e43-90dc1e7d28b9 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:06.472407 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c3deb-7f1a-4c09-8e43-90dc1e7d28b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:06.490976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-cb8c3deb-7f1a-4c09-8e43-90dc1e7d28b9 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:06.492467 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-cb8c3deb-7f1a-4c09-8e43-90dc1e7d28b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:06.550264 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1321/2634] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:02:06.558880 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3622736-de21-4a1f-9358-6a8d7bf42b9e None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:06.559182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3622736-de21-4a1f-9358-6a8d7bf42b9e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:06.559567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3622736-de21-4a1f-9358-6a8d7bf42b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:06.578738 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a3622736-de21-4a1f-9358-6a8d7bf42b9e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:06.580034 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a3622736-de21-4a1f-9358-6a8d7bf42b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:06.588156 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1314/2635] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:02:06.592807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c3deb-7f1a-4c09-8e43-90dc1e7d28b9 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:06.602843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c3deb-7f1a-4c09-8e43-90dc1e7d28b9 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:06.608641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fe975-88aa-43bb-9400-db78d06669fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:06.609295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fe975-88aa-43bb-9400-db78d06669fc admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:06.609645 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fe975-88aa-43bb-9400-db78d06669fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:06.609927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fe975-88aa-43bb-9400-db78d06669fc admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:06.610660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fe975-88aa-43bb-9400-db78d06669fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:02:06.610995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fe975-88aa-43bb-9400-db78d06669fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:06.612631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fe975-88aa-43bb-9400-db78d06669fc admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:06.624254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1322/2636] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 14:02:06.628717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3622736-de21-4a1f-9358-6a8d7bf42b9e None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:06.638930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3622736-de21-4a1f-9358-6a8d7bf42b9e None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:06.644377 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26617b59-7bbe-4142-bde9-20af5f9596a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:06.645051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26617b59-7bbe-4142-bde9-20af5f9596a9 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:06.645334 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26617b59-7bbe-4142-bde9-20af5f9596a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:06.645608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26617b59-7bbe-4142-bde9-20af5f9596a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:06.646298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26617b59-7bbe-4142-bde9-20af5f9596a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:06.648652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26617b59-7bbe-4142-bde9-20af5f9596a9 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:06.675683 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1315/2637] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:02:06 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:02:06.679991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fe975-88aa-43bb-9400-db78d06669fc admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:06.690184 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fe975-88aa-43bb-9400-db78d06669fc admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:06.719538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:06.749522 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec8753-86a6-48d3-ab7a-fa8ab4c66924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:06.750186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec8753-86a6-48d3-ab7a-fa8ab4c66924 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:06.750473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec8753-86a6-48d3-ab7a-fa8ab4c66924 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:06.750753 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec8753-86a6-48d3-ab7a-fa8ab4c66924 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:06.751479 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec8753-86a6-48d3-ab7a-fa8ab4c66924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:06.753390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec8753-86a6-48d3-ab7a-fa8ab4c66924 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:06.772748 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1323/2638] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:02:06 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:02:06.776782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26617b59-7bbe-4142-bde9-20af5f9596a9 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:06.787737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26617b59-7bbe-4142-bde9-20af5f9596a9 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:06.792297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aac4eb-c537-472a-96cc-48c148dbb814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:06.792951 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aac4eb-c537-472a-96cc-48c148dbb814 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:06.793233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aac4eb-c537-472a-96cc-48c148dbb814 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:06.793519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aac4eb-c537-472a-96cc-48c148dbb814 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:06.795132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aac4eb-c537-472a-96cc-48c148dbb814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:06.796698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aac4eb-c537-472a-96cc-48c148dbb814 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:06.806820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1316/2639] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:02:06.810789 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec8753-86a6-48d3-ab7a-fa8ab4c66924 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:06.820427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec8753-86a6-48d3-ab7a-fa8ab4c66924 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:06.825577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9e5aa-11f3-4bf0-b1e8-3ad3491f17c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:06.826290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9e5aa-11f3-4bf0-b1e8-3ad3491f17c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:06.826569 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9e5aa-11f3-4bf0-b1e8-3ad3491f17c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:06.826843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9e5aa-11f3-4bf0-b1e8-3ad3491f17c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:06.827930 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9e5aa-11f3-4bf0-b1e8-3ad3491f17c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:06.829510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9e5aa-11f3-4bf0-b1e8-3ad3491f17c9 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:06.839323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1324/2640] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:02:06.843951 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aac4eb-c537-472a-96cc-48c148dbb814 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:06.853671 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aac4eb-c537-472a-96cc-48c148dbb814 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:06.858424 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf74973-01d3-4ea2-95a0-4a3a0e1b0474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:06.859315 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf74973-01d3-4ea2-95a0-4a3a0e1b0474 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:06.859621 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf74973-01d3-4ea2-95a0-4a3a0e1b0474 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:06.859895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf74973-01d3-4ea2-95a0-4a3a0e1b0474 admin admin] PATH_INFO: `/v3/projects/07f1a812f61847fdba5ff1a8586ad767/users/c6a8830efe9348689be487ad9b49e1c9/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:06.874003 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf74973-01d3-4ea2-95a0-4a3a0e1b0474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f1a812f61847fdba5ff1a8586ad767, user_id=c6a8830efe9348689be487ad9b49e1c9, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:06.876648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf74973-01d3-4ea2-95a0-4a3a0e1b0474 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:06.886492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:06.891916 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1317/2641] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:02:06 2022] PUT /identity/v3/projects/07f1a812f61847fdba5ff1a8586ad767/users/c6a8830efe9348689be487ad9b49e1c9/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:06.904545 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afe5d7-7bc8-4e36-a288-855eb0563864 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:06.904837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afe5d7-7bc8-4e36-a288-855eb0563864 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:06.905149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afe5d7-7bc8-4e36-a288-855eb0563864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:06.933158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-d9afe5d7-7bc8-4e36-a288-855eb0563864 None None] MFA Rules not processed for user `c6a8830efe9348689be487ad9b49e1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:06.934355 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-d9afe5d7-7bc8-4e36-a288-855eb0563864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:07.004025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1325/2642] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:02:07.016167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf74973-01d3-4ea2-95a0-4a3a0e1b0474 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:07.018048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5bf74973-01d3-4ea2-95a0-4a3a0e1b0474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:07.087601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf74973-01d3-4ea2-95a0-4a3a0e1b0474 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:07.091860 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db80876-1625-4352-8fef-5a7760c5b1c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:07.092441 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db80876-1625-4352-8fef-5a7760c5b1c4 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:07.092722 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db80876-1625-4352-8fef-5a7760c5b1c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:07.092998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db80876-1625-4352-8fef-5a7760c5b1c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:07.101843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db80876-1625-4352-8fef-5a7760c5b1c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:07.103306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db80876-1625-4352-8fef-5a7760c5b1c4 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:07.112516 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1318/2643] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:02:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:02:17.000832 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:17.108717 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=26 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:28.848090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afe5d7-7bc8-4e36-a288-855eb0563864 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:28.909627 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afe5d7-7bc8-4e36-a288-855eb0563864 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:28.914328 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:28.915494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:28.915786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:28.916064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] PATH_INFO: `/v3/users/c6a8830efe9348689be487ad9b49e1c9` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:28.917249 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a8830efe9348689be487ad9b49e1c9)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:28.919670 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:28.966506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a8830efe9348689be487ad9b49e1c9'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:28.974072 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a8830efe9348689be487ad9b49e1c9'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:28.981106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a8830efe9348689be487ad9b49e1c9'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:28.981781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a8830efe9348689be487ad9b49e1c9'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:28.982393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a8830efe9348689be487ad9b49e1c9'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:29.000061 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1326/2644] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:02:28 2022] DELETE /identity/v3/users/c6a8830efe9348689be487ad9b49e1c9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:29.004647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e85b66-dce0-492c-871c-c999afe07662 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:29.005143 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e85b66-dce0-492c-871c-c999afe07662 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:29.005251 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e85b66-dce0-492c-871c-c999afe07662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:29.023782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-67e85b66-dce0-492c-871c-c999afe07662 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:29.025148 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-67e85b66-dce0-492c-871c-c999afe07662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:29.084994 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1319/2645] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:02:29.094710 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:29.096151 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:29.153559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a98518-8412-47d3-b74e-914250742355 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:29.157150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8057b-4b75-40ee-ad44-ef12ef0201e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:29.157781 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8057b-4b75-40ee-ad44-ef12ef0201e5 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:29.158064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8057b-4b75-40ee-ad44-ef12ef0201e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:29.158435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8057b-4b75-40ee-ad44-ef12ef0201e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:29.167428 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8057b-4b75-40ee-ad44-ef12ef0201e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:29.168944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8057b-4b75-40ee-ad44-ef12ef0201e5 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:29.178138 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1327/2646] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:02:29.441752 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e85b66-dce0-492c-871c-c999afe07662 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:29.443216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-67e85b66-dce0-492c-871c-c999afe07662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:29.457638 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e85b66-dce0-492c-871c-c999afe07662 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:29.462319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:29.463300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:29.463588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:29.463868 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f admin admin] PATH_INFO: `/v3/projects/07f1a812f61847fdba5ff1a8586ad767` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:29.465187 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=07f1a812f61847fdba5ff1a8586ad767)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:29.467660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:29.514411 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07f1a812f61847fdba5ff1a8586ad767', 'request_id': 'req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:29.520372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07f1a812f61847fdba5ff1a8586ad767', 'request_id': 'req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:29.528275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07f1a812f61847fdba5ff1a8586ad767', 'request_id': 'req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:29.538247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f admin admin] The token cache is being invalidate because project 07f1a812f61847fdba5ff1a8586ad767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:02:29.538530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:29.539161 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6ef7b738-70ba-446f-8f11-d6342d2bdc9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:29.540697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1320/2647] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:02:29 2022] DELETE /identity/v3/projects/07f1a812f61847fdba5ff1a8586ad767 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:29.545610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47b62d-b311-4664-a20c-d19470617e6d None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:29.545917 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47b62d-b311-4664-a20c-d19470617e6d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:29.546182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47b62d-b311-4664-a20c-d19470617e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:29.565185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-8b47b62d-b311-4664-a20c-d19470617e6d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:29.566372 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8b47b62d-b311-4664-a20c-d19470617e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:29.583125 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:29.626595 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1328/2648] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:02:29.634764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7eb467-ebef-4576-8493-e9f0d3abe928 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:29.635089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7eb467-ebef-4576-8493-e9f0d3abe928 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:29.635377 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7eb467-ebef-4576-8493-e9f0d3abe928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:29.654960 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-9b7eb467-ebef-4576-8493-e9f0d3abe928 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:29.656159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9b7eb467-ebef-4576-8493-e9f0d3abe928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:29.664132 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1321/2649] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:02:29.668516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47b62d-b311-4664-a20c-d19470617e6d None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:29.678472 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47b62d-b311-4664-a20c-d19470617e6d None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:29.683172 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6377a-8e2f-4d67-a885-37316c6a95bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:29.683914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6377a-8e2f-4d67-a885-37316c6a95bf admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:29.684265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6377a-8e2f-4d67-a885-37316c6a95bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:29.684574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6377a-8e2f-4d67-a885-37316c6a95bf admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:29.685448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6377a-8e2f-4d67-a885-37316c6a95bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:02:29.685829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6377a-8e2f-4d67-a885-37316c6a95bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:29.687557 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6377a-8e2f-4d67-a885-37316c6a95bf admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:29.700968 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1329/2650] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:02:29.704550 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7eb467-ebef-4576-8493-e9f0d3abe928 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:29.713610 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7eb467-ebef-4576-8493-e9f0d3abe928 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:29.719074 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbae4f2-054c-4202-a319-bd492229ee74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:29.719779 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbae4f2-054c-4202-a319-bd492229ee74 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:29.720059 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbae4f2-054c-4202-a319-bd492229ee74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:29.720333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbae4f2-054c-4202-a319-bd492229ee74 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:29.721009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbae4f2-054c-4202-a319-bd492229ee74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:29.722934 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbae4f2-054c-4202-a319-bd492229ee74 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:29.749597 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1322/2651] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:02:29 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:02:29.753819 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6377a-8e2f-4d67-a885-37316c6a95bf admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:29.766188 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6377a-8e2f-4d67-a885-37316c6a95bf admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:29.821019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd677c3-9997-4799-82d4-e6b49ad6fcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:29.821600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd677c3-9997-4799-82d4-e6b49ad6fcdf admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:29.821884 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd677c3-9997-4799-82d4-e6b49ad6fcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:29.822160 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd677c3-9997-4799-82d4-e6b49ad6fcdf admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:29.822848 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd677c3-9997-4799-82d4-e6b49ad6fcdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:29.825516 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd677c3-9997-4799-82d4-e6b49ad6fcdf admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:29.844873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1330/2652] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:02:29 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:02:29.849122 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbae4f2-054c-4202-a319-bd492229ee74 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:29.859241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbae4f2-054c-4202-a319-bd492229ee74 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:29.863791 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96673d-8c0e-415f-a471-ac90125e76bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:29.864445 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96673d-8c0e-415f-a471-ac90125e76bc admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:29.864726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96673d-8c0e-415f-a471-ac90125e76bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:29.865002 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96673d-8c0e-415f-a471-ac90125e76bc admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:29.865716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96673d-8c0e-415f-a471-ac90125e76bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:29.867336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96673d-8c0e-415f-a471-ac90125e76bc admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:29.878293 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1323/2653] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:02:29.882394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd677c3-9997-4799-82d4-e6b49ad6fcdf admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:29.893107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd677c3-9997-4799-82d4-e6b49ad6fcdf admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:29.897768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0809846-4a6c-4c0d-be13-4e7078ab5ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:29.898429 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0809846-4a6c-4c0d-be13-4e7078ab5ead admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:29.898700 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0809846-4a6c-4c0d-be13-4e7078ab5ead admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:29.899016 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0809846-4a6c-4c0d-be13-4e7078ab5ead admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:29.899741 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0809846-4a6c-4c0d-be13-4e7078ab5ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:29.901309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0809846-4a6c-4c0d-be13-4e7078ab5ead admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:29.911347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1331/2654] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:02:29.915540 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96673d-8c0e-415f-a471-ac90125e76bc admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:29.925423 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96673d-8c0e-415f-a471-ac90125e76bc admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:29.930043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83448d-d790-478e-947f-cdd7639bfce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:29.930942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83448d-d790-478e-947f-cdd7639bfce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:29.931226 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83448d-d790-478e-947f-cdd7639bfce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:29.931502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83448d-d790-478e-947f-cdd7639bfce3 admin admin] PATH_INFO: `/v3/projects/56a64efcc2324083b99ab916bd2a2ce2/users/82bb54f3f3f940e594f9457bd2cde9e8/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:29.946398 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83448d-d790-478e-947f-cdd7639bfce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56a64efcc2324083b99ab916bd2a2ce2, user_id=82bb54f3f3f940e594f9457bd2cde9e8, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:29.949060 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83448d-d790-478e-947f-cdd7639bfce3 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:29.962497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1324/2655] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:02:29 2022] PUT /identity/v3/projects/56a64efcc2324083b99ab916bd2a2ce2/users/82bb54f3f3f940e594f9457bd2cde9e8/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:29.973878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59851750-3f08-4c30-bc76-47c12eaca222 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:29.974184 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59851750-3f08-4c30-bc76-47c12eaca222 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:29.974466 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59851750-3f08-4c30-bc76-47c12eaca222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:30.001149 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-59851750-3f08-4c30-bc76-47c12eaca222 None None] MFA Rules not processed for user `82bb54f3f3f940e594f9457bd2cde9e8`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:30.002398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-59851750-3f08-4c30-bc76-47c12eaca222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:30.067060 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1332/2656] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:02:30.078624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83448d-d790-478e-947f-cdd7639bfce3 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:30.080340 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ca83448d-d790-478e-947f-cdd7639bfce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:30.101596 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:30.139845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83448d-d790-478e-947f-cdd7639bfce3 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:30.143501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b899a29-bb93-4e76-b94b-619f62cfc445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:30.144135 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b899a29-bb93-4e76-b94b-619f62cfc445 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:30.144417 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b899a29-bb93-4e76-b94b-619f62cfc445 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:30.144693 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b899a29-bb93-4e76-b94b-619f62cfc445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:30.152912 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b899a29-bb93-4e76-b94b-619f62cfc445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:30.154339 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b899a29-bb93-4e76-b94b-619f62cfc445 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:30.162930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1325/2657] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:02:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:02:30.318738 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59851750-3f08-4c30-bc76-47c12eaca222 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:30.320382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-59851750-3f08-4c30-bc76-47c12eaca222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:30.341116 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:30.384995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59851750-3f08-4c30-bc76-47c12eaca222 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:30.392454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89f87c-a74f-4e05-8a21-b1c50ad64b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:30.393303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89f87c-a74f-4e05-8a21-b1c50ad64b56 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:30.393616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89f87c-a74f-4e05-8a21-b1c50ad64b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:30.393902 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89f87c-a74f-4e05-8a21-b1c50ad64b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:30.399098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5c89f87c-a74f-4e05-8a21-b1c50ad64b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:30.467109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89f87c-a74f-4e05-8a21-b1c50ad64b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:30.468560 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89f87c-a74f-4e05-8a21-b1c50ad64b56 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:30.477879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1333/2658] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:02:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:02:37.832259 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b899a29-bb93-4e76-b94b-619f62cfc445 service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:37.842891 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b899a29-bb93-4e76-b94b-619f62cfc445 service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:37.847615 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb08d1b-000f-4bcc-adb3-ee4f4796729b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:37.852352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb08d1b-000f-4bcc-adb3-ee4f4796729b service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:37.852352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb08d1b-000f-4bcc-adb3-ee4f4796729b service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:37.852352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb08d1b-000f-4bcc-adb3-ee4f4796729b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:37.852352 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3eb08d1b-000f-4bcc-adb3-ee4f4796729b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:37.944313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb08d1b-000f-4bcc-adb3-ee4f4796729b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:37.945946 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb08d1b-000f-4bcc-adb3-ee4f4796729b service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:37.956883 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1326/2659] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:02:40.473567 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=40 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:44.196433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89f87c-a74f-4e05-8a21-b1c50ad64b56 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:44.197964 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5c89f87c-a74f-4e05-8a21-b1c50ad64b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:44.219674 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89f87c-a74f-4e05-8a21-b1c50ad64b56 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:44.224390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:44.225274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:44.225561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:44.225849 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] PATH_INFO: `/v3/users/d5fabcc754684ded96c800be75777553` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:44.227083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5fabcc754684ded96c800be75777553)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:44.229260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:44.280969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5fabcc754684ded96c800be75777553'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.287789 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5fabcc754684ded96c800be75777553'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.295623 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5fabcc754684ded96c800be75777553'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.296271 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5fabcc754684ded96c800be75777553'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.296872 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5fabcc754684ded96c800be75777553'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.314791 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1334/2660] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:02:44 2022] DELETE /identity/v3/users/d5fabcc754684ded96c800be75777553 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:44.485736 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb08d1b-000f-4bcc-adb3-ee4f4796729b service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:44.496646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb08d1b-000f-4bcc-adb3-ee4f4796729b service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:44.557022 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:44.557812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:44.558093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:44.558363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] PATH_INFO: `/v3/projects/0197b4cff99649ef9da1e77b1461dd76` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:44.559566 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0197b4cff99649ef9da1e77b1461dd76)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:44.563491 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:44.609648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0197b4cff99649ef9da1e77b1461dd76', 'request_id': 'req-b5ccd536-1e85-466a-8180-f9ddeac96d07'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.615602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0197b4cff99649ef9da1e77b1461dd76', 'request_id': 'req-b5ccd536-1e85-466a-8180-f9ddeac96d07'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.619090 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:44.624096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0197b4cff99649ef9da1e77b1461dd76', 'request_id': 'req-b5ccd536-1e85-466a-8180-f9ddeac96d07'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.636973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] The token cache is being invalidate because project 0197b4cff99649ef9da1e77b1461dd76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:02:44.637265 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.638075 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.639446 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1327/2661] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:02:44 2022] DELETE /identity/v3/projects/0197b4cff99649ef9da1e77b1461dd76 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:02:44.643383 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:44.644788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:44.658001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4c9cf-e9ef-4314-bd3e-1994adbf9315 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:44.715394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0cb94-b23c-4c58-8188-765d66781092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:44.716771 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0cb94-b23c-4c58-8188-765d66781092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:44.717182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0cb94-b23c-4c58-8188-765d66781092 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:44.717465 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0cb94-b23c-4c58-8188-765d66781092 admin admin] PATH_INFO: `/v3/users/b0d9ebf3b3d046e79f6032573cc0a2bf` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:44.718642 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0cb94-b23c-4c58-8188-765d66781092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0d9ebf3b3d046e79f6032573cc0a2bf)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:44.720577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0cb94-b23c-4c58-8188-765d66781092 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:44.766637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-36e0cb94-b23c-4c58-8188-765d66781092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0d9ebf3b3d046e79f6032573cc0a2bf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.774162 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-36e0cb94-b23c-4c58-8188-765d66781092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0d9ebf3b3d046e79f6032573cc0a2bf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.781769 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-36e0cb94-b23c-4c58-8188-765d66781092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0d9ebf3b3d046e79f6032573cc0a2bf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.782406 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-36e0cb94-b23c-4c58-8188-765d66781092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0d9ebf3b3d046e79f6032573cc0a2bf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.783041 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-36e0cb94-b23c-4c58-8188-765d66781092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0d9ebf3b3d046e79f6032573cc0a2bf'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:44.790742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:44.802663 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1335/2662] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:02:44 2022] DELETE /identity/v3/users/b0d9ebf3b3d046e79f6032573cc0a2bf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:44.976958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:44.978536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:44.999353 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccd536-1e85-466a-8180-f9ddeac96d07 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:45.057276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e9b4e-0827-40da-bd47-459a6c938da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:45.058140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e9b4e-0827-40da-bd47-459a6c938da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:45.058427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e9b4e-0827-40da-bd47-459a6c938da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:45.058705 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e9b4e-0827-40da-bd47-459a6c938da6 admin admin] PATH_INFO: `/v3/projects/f70c3715c82e42958ab6bb02373acc2c` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:45.059947 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e9b4e-0827-40da-bd47-459a6c938da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f70c3715c82e42958ab6bb02373acc2c)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:45.061842 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e9b4e-0827-40da-bd47-459a6c938da6 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:45.114941 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2e9e9b4e-0827-40da-bd47-459a6c938da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f70c3715c82e42958ab6bb02373acc2c', 'request_id': 'req-2e9e9b4e-0827-40da-bd47-459a6c938da6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:45.121333 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2e9e9b4e-0827-40da-bd47-459a6c938da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f70c3715c82e42958ab6bb02373acc2c', 'request_id': 'req-2e9e9b4e-0827-40da-bd47-459a6c938da6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:45.128928 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2e9e9b4e-0827-40da-bd47-459a6c938da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f70c3715c82e42958ab6bb02373acc2c', 'request_id': 'req-2e9e9b4e-0827-40da-bd47-459a6c938da6'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:45.143216 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2e9e9b4e-0827-40da-bd47-459a6c938da6 admin admin] The token cache is being invalidate because project f70c3715c82e42958ab6bb02373acc2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:02:45.143521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2e9e9b4e-0827-40da-bd47-459a6c938da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:45.144319 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-2e9e9b4e-0827-40da-bd47-459a6c938da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:02:45.150777 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1328/2663] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:02:44 2022] DELETE /identity/v3/projects/f70c3715c82e42958ab6bb02373acc2c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:45.152632 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3751d0-7496-404c-887b-d4679782ac67 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:45.152951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3751d0-7496-404c-887b-d4679782ac67 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:45.153250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3751d0-7496-404c-887b-d4679782ac67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:45.178907 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ce3751d0-7496-404c-887b-d4679782ac67 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:45.180109 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ce3751d0-7496-404c-887b-d4679782ac67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:45.247020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1336/2664] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:02:45.255168 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2ed51-9bd9-4f4d-8f52-40a5fb7e1022 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:45.255460 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2ed51-9bd9-4f4d-8f52-40a5fb7e1022 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:45.255740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2ed51-9bd9-4f4d-8f52-40a5fb7e1022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:45.275301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-70d2ed51-9bd9-4f4d-8f52-40a5fb7e1022 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:45.276533 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-70d2ed51-9bd9-4f4d-8f52-40a5fb7e1022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:45.285876 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1329/2665] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:02:45.290206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3751d0-7496-404c-887b-d4679782ac67 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:45.302731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3751d0-7496-404c-887b-d4679782ac67 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:45.311321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8749ef-6db5-42e8-b225-c40a3bbb10c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:45.311321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8749ef-6db5-42e8-b225-c40a3bbb10c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:45.311321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8749ef-6db5-42e8-b225-c40a3bbb10c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:45.311321 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8749ef-6db5-42e8-b225-c40a3bbb10c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:45.311715 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8749ef-6db5-42e8-b225-c40a3bbb10c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:02:45.311715 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8749ef-6db5-42e8-b225-c40a3bbb10c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:45.311908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8749ef-6db5-42e8-b225-c40a3bbb10c3 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:45.330697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1337/2666] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:02:45.331247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2ed51-9bd9-4f4d-8f52-40a5fb7e1022 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:45.341048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2ed51-9bd9-4f4d-8f52-40a5fb7e1022 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:45.347214 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdff74-3e5a-4259-b7e4-9a3d0600f172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:45.348567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdff74-3e5a-4259-b7e4-9a3d0600f172 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:45.348567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdff74-3e5a-4259-b7e4-9a3d0600f172 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:45.348567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdff74-3e5a-4259-b7e4-9a3d0600f172 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:45.349096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdff74-3e5a-4259-b7e4-9a3d0600f172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:45.351114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdff74-3e5a-4259-b7e4-9a3d0600f172 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:45.385311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1330/2667] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:02:45 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:02:45.387250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8749ef-6db5-42e8-b225-c40a3bbb10c3 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:45.398948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8749ef-6db5-42e8-b225-c40a3bbb10c3 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:45.460539 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760aa8e-89cb-4598-9387-c51657685d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:45.461203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760aa8e-89cb-4598-9387-c51657685d42 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:45.461485 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760aa8e-89cb-4598-9387-c51657685d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:45.462206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760aa8e-89cb-4598-9387-c51657685d42 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:45.462440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760aa8e-89cb-4598-9387-c51657685d42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:45.464376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760aa8e-89cb-4598-9387-c51657685d42 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:45.486706 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1338/2668] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:02:45 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:02:45.490728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdff74-3e5a-4259-b7e4-9a3d0600f172 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:45.501030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdff74-3e5a-4259-b7e4-9a3d0600f172 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:45.506584 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d17e81-0221-4b66-998f-71c1760b4cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:45.508380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d17e81-0221-4b66-998f-71c1760b4cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:45.508756 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d17e81-0221-4b66-998f-71c1760b4cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:45.509239 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d17e81-0221-4b66-998f-71c1760b4cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:45.510026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d17e81-0221-4b66-998f-71c1760b4cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:45.511887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d17e81-0221-4b66-998f-71c1760b4cd0 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:45.523383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1331/2669] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:02:45.527559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760aa8e-89cb-4598-9387-c51657685d42 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:45.539041 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760aa8e-89cb-4598-9387-c51657685d42 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:45.545461 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28fa24-ec5e-4640-b193-175777e3fd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:45.546203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28fa24-ec5e-4640-b193-175777e3fd75 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:45.546652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28fa24-ec5e-4640-b193-175777e3fd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:45.547498 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28fa24-ec5e-4640-b193-175777e3fd75 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:45.548050 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28fa24-ec5e-4640-b193-175777e3fd75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:45.549994 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28fa24-ec5e-4640-b193-175777e3fd75 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:45.567944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1339/2670] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:02:45.572222 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d17e81-0221-4b66-998f-71c1760b4cd0 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:45.585547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d17e81-0221-4b66-998f-71c1760b4cd0 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:45.590529 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f0f10-8bac-4e46-87aa-ac18ef44c833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:45.591809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f0f10-8bac-4e46-87aa-ac18ef44c833 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:45.592154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f0f10-8bac-4e46-87aa-ac18ef44c833 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:45.592427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f0f10-8bac-4e46-87aa-ac18ef44c833 admin admin] PATH_INFO: `/v3/projects/6d87d3963b1b4fb29e09d8f72f64fb6e/users/df54f67274d24463961e1f48dcb37447/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:45.621231 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f0f10-8bac-4e46-87aa-ac18ef44c833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d87d3963b1b4fb29e09d8f72f64fb6e, user_id=df54f67274d24463961e1f48dcb37447, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:45.624981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f0f10-8bac-4e46-87aa-ac18ef44c833 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:45.649769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1332/2671] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:02:45 2022] PUT /identity/v3/projects/6d87d3963b1b4fb29e09d8f72f64fb6e/users/df54f67274d24463961e1f48dcb37447/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:45.655495 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793fd27-8a1a-4089-aa9b-8bceaf357cad None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:45.655787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793fd27-8a1a-4089-aa9b-8bceaf357cad None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:45.656093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793fd27-8a1a-4089-aa9b-8bceaf357cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:45.675515 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-6793fd27-8a1a-4089-aa9b-8bceaf357cad None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:45.677346 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-6793fd27-8a1a-4089-aa9b-8bceaf357cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:45.734333 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:45.747427 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1340/2672] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:02:45.757435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f0f10-8bac-4e46-87aa-ac18ef44c833 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:45.759375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3a3f0f10-8bac-4e46-87aa-ac18ef44c833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:45.779478 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:45.836368 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f0f10-8bac-4e46-87aa-ac18ef44c833 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:45.841853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542be70-278d-4f2f-9c77-4e062ed067ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:45.842427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542be70-278d-4f2f-9c77-4e062ed067ef service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:45.842716 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542be70-278d-4f2f-9c77-4e062ed067ef service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:45.843220 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542be70-278d-4f2f-9c77-4e062ed067ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:45.859781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542be70-278d-4f2f-9c77-4e062ed067ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:45.861234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542be70-278d-4f2f-9c77-4e062ed067ef service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:45.891061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1333/2673] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:02:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:02:46.029304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793fd27-8a1a-4089-aa9b-8bceaf357cad None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:46.031894 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-6793fd27-8a1a-4089-aa9b-8bceaf357cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:46.116559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793fd27-8a1a-4089-aa9b-8bceaf357cad None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:46.121591 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b9569-c0da-41c8-8ca3-41ba6b2a1f86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b105f8163ce84a9ba725e6b04d348446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:46.122247 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b9569-c0da-41c8-8ca3-41ba6b2a1f86 service placement] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:46.122618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b9569-c0da-41c8-8ca3-41ba6b2a1f86 service placement] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:46.122979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b9569-c0da-41c8-8ca3-41ba6b2a1f86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:46.124724 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a30b9569-c0da-41c8-8ca3-41ba6b2a1f86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:46.137707 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b9569-c0da-41c8-8ca3-41ba6b2a1f86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:46.140205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b9569-c0da-41c8-8ca3-41ba6b2a1f86 service placement] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:46.160373 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1341/2674] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:02:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:02:50.146480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ced47-9b0c-4179-a943-3f4bea304cea None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:50.147093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ced47-9b0c-4179-a943-3f4bea304cea None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:50.147093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ced47-9b0c-4179-a943-3f4bea304cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:50.177066 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-625ced47-9b0c-4179-a943-3f4bea304cea None None] MFA Rules not processed for user `df54f67274d24463961e1f48dcb37447`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:50.178348 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-625ced47-9b0c-4179-a943-3f4bea304cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:50.261540 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1334/2675] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:02:50.267185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b9569-c0da-41c8-8ca3-41ba6b2a1f86 service placement] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:50.279147 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b9569-c0da-41c8-8ca3-41ba6b2a1f86 service placement] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:50.284578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91398ca2-7bb0-423a-9d4a-cded0e8c3403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:50.285339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398ca2-7bb0-423a-9d4a-cded0e8c3403 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:50.285756 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398ca2-7bb0-423a-9d4a-cded0e8c3403 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:50.285862 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398ca2-7bb0-423a-9d4a-cded0e8c3403 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:50.286611 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91398ca2-7bb0-423a-9d4a-cded0e8c3403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:50.288610 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91398ca2-7bb0-423a-9d4a-cded0e8c3403 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:50.316686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1342/2676] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:02:50 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:02:50.320769 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ced47-9b0c-4179-a943-3f4bea304cea None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:50.332300 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ced47-9b0c-4179-a943-3f4bea304cea None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:50.405212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4acde-0958-4f55-b503-c3a5943058cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:50.406443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4acde-0958-4f55-b503-c3a5943058cc admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:50.406764 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4acde-0958-4f55-b503-c3a5943058cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:50.407131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4acde-0958-4f55-b503-c3a5943058cc admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:50.408089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4acde-0958-4f55-b503-c3a5943058cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:50.410117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4acde-0958-4f55-b503-c3a5943058cc admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:50.436572 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1335/2677] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:02:50 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:02:50.441187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91398ca2-7bb0-423a-9d4a-cded0e8c3403 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:50.452177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91398ca2-7bb0-423a-9d4a-cded0e8c3403 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:50.457315 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518ec3c-5ea5-4a6d-92e2-b158ad5517f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:50.458226 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518ec3c-5ea5-4a6d-92e2-b158ad5517f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:50.458518 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518ec3c-5ea5-4a6d-92e2-b158ad5517f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:50.459001 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518ec3c-5ea5-4a6d-92e2-b158ad5517f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:50.459777 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518ec3c-5ea5-4a6d-92e2-b158ad5517f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:50.462767 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518ec3c-5ea5-4a6d-92e2-b158ad5517f1 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:50.477372 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1343/2678] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:02:50.481391 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4acde-0958-4f55-b503-c3a5943058cc admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:50.493924 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4acde-0958-4f55-b503-c3a5943058cc admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:50.500473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a478add-56e8-4463-b612-c3cb7e88bf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:50.501397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a478add-56e8-4463-b612-c3cb7e88bf1a admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:50.501681 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a478add-56e8-4463-b612-c3cb7e88bf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:50.501958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a478add-56e8-4463-b612-c3cb7e88bf1a admin admin] PATH_INFO: `/v3/projects/6eea54f5ac214cbeac589e8af91b3a84/users/c8e8c5e2850d4274aa09fde41409b8d6/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:50.541152 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a478add-56e8-4463-b612-c3cb7e88bf1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eea54f5ac214cbeac589e8af91b3a84, user_id=c8e8c5e2850d4274aa09fde41409b8d6, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:50.543978 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a478add-56e8-4463-b612-c3cb7e88bf1a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:50.560523 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1336/2679] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:02:50 2022] PUT /identity/v3/projects/6eea54f5ac214cbeac589e8af91b3a84/users/c8e8c5e2850d4274aa09fde41409b8d6/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:50.564274 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518ec3c-5ea5-4a6d-92e2-b158ad5517f1 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:50.574604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518ec3c-5ea5-4a6d-92e2-b158ad5517f1 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:50.646942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2167d-ffb1-46f8-a804-e9734eb24a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:50.647757 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2167d-ffb1-46f8-a804-e9734eb24a2f admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:50.648040 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2167d-ffb1-46f8-a804-e9734eb24a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:50.648317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2167d-ffb1-46f8-a804-e9734eb24a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:50.649040 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2167d-ffb1-46f8-a804-e9734eb24a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:50.650625 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2167d-ffb1-46f8-a804-e9734eb24a2f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:50.661601 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1344/2680] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:02:50.665311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a478add-56e8-4463-b612-c3cb7e88bf1a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:50.676301 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a478add-56e8-4463-b612-c3cb7e88bf1a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:50.681277 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eee4d0-c287-431d-a1be-e9a2f9c28651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:50.682178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eee4d0-c287-431d-a1be-e9a2f9c28651 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:50.682473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eee4d0-c287-431d-a1be-e9a2f9c28651 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:50.682753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eee4d0-c287-431d-a1be-e9a2f9c28651 admin admin] PATH_INFO: `/v3/projects/6eea54f5ac214cbeac589e8af91b3a84/users/c8e8c5e2850d4274aa09fde41409b8d6/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:50.685471 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eee4d0-c287-431d-a1be-e9a2f9c28651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eea54f5ac214cbeac589e8af91b3a84, user_id=c8e8c5e2850d4274aa09fde41409b8d6, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:50.688208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eee4d0-c287-431d-a1be-e9a2f9c28651 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:50.693965 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:50.701050 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1337/2681] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:02:50 2022] PUT /identity/v3/projects/6eea54f5ac214cbeac589e8af91b3a84/users/c8e8c5e2850d4274aa09fde41409b8d6/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:02:55.908326 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d75b7d-36a3-404e-820d-6bff3c469e59 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:55.908906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d75b7d-36a3-404e-820d-6bff3c469e59 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:55.908906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d75b7d-36a3-404e-820d-6bff3c469e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:55.939503 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-04d75b7d-36a3-404e-820d-6bff3c469e59 None None] MFA Rules not processed for user `c8e8c5e2850d4274aa09fde41409b8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:55.941444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-04d75b7d-36a3-404e-820d-6bff3c469e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:55.994424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:02:56.031087 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1345/2682] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:02:56.045813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eee4d0-c287-431d-a1be-e9a2f9c28651 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:02:56.047753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-39eee4d0-c287-431d-a1be-e9a2f9c28651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:56.126565 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eee4d0-c287-431d-a1be-e9a2f9c28651 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:02:56.132329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f6ffd-ca4a-48cb-a645-00bc5b2d4390 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:02:56.132983 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f6ffd-ca4a-48cb-a645-00bc5b2d4390 service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:56.133264 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f6ffd-ca4a-48cb-a645-00bc5b2d4390 service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:56.133544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f6ffd-ca4a-48cb-a645-00bc5b2d4390 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:56.186952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b43e9-1d56-427d-8b2c-d91ab2ba5060 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:02:56.187261 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b43e9-1d56-427d-8b2c-d91ab2ba5060 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:02:56.187702 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b43e9-1d56-427d-8b2c-d91ab2ba5060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:02:56.199638 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f6ffd-ca4a-48cb-a645-00bc5b2d4390 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:02:56.202040 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f6ffd-ca4a-48cb-a645-00bc5b2d4390 service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:02:56.206606 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-ec4b43e9-1d56-427d-8b2c-d91ab2ba5060 None None] MFA Rules not processed for user `12d3af57f5e34f798336d353a111cefd`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:02:56.207948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ec4b43e9-1d56-427d-8b2c-d91ab2ba5060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:02:56.211793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1338/2683] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:02:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:02:56.293173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1346/2684] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:02:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:03:06.208036 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=57 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:03:06.285839 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=36 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:03:06.386612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5ab17-8ccd-4fc7-81fd-5c003411ee7e None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:06.386930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5ab17-8ccd-4fc7-81fd-5c003411ee7e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:06.387206 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5ab17-8ccd-4fc7-81fd-5c003411ee7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:06.405947 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-16f5ab17-8ccd-4fc7-81fd-5c003411ee7e None None] MFA Rules not processed for user `ab1b34a915464792b5d3b74956bc49a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:03:06.407185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-16f5ab17-8ccd-4fc7-81fd-5c003411ee7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:06.462485 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1339/2685] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:03:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:03:12.354908 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b43e9-1d56-427d-8b2c-d91ab2ba5060 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:12.356358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ec4b43e9-1d56-427d-8b2c-d91ab2ba5060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:12.421032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b43e9-1d56-427d-8b2c-d91ab2ba5060 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:12.425948 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:12.426817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:12.427176 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:12.427454 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] PATH_INFO: `/v3/users/82bb54f3f3f940e594f9457bd2cde9e8` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:12.428690 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] RBAC: Authorizing `identity:delete_user(user_id=82bb54f3f3f940e594f9457bd2cde9e8)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:12.430644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:12.482598 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82bb54f3f3f940e594f9457bd2cde9e8'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:12.490359 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82bb54f3f3f940e594f9457bd2cde9e8'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:12.497861 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82bb54f3f3f940e594f9457bd2cde9e8'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:12.498503 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82bb54f3f3f940e594f9457bd2cde9e8'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:12.499150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82bb54f3f3f940e594f9457bd2cde9e8'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:12.519000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1347/2686] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:03:12 2022] DELETE /identity/v3/users/82bb54f3f3f940e594f9457bd2cde9e8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:03:12.523590 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6188c-8950-41a6-9014-08b159fa3db8 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:12.524075 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6188c-8950-41a6-9014-08b159fa3db8 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:12.524298 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6188c-8950-41a6-9014-08b159fa3db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:12.542692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a1e6188c-8950-41a6-9014-08b159fa3db8 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:03:12.544011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a1e6188c-8950-41a6-9014-08b159fa3db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:12.604362 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1340/2687] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:03:12.614846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:12.616263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:12.677391 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7158256-8799-429f-83dc-ccbeec43891d admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:12.681055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61b904-44a3-4d71-b809-958981901cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:12.681694 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61b904-44a3-4d71-b809-958981901cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:12.682002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61b904-44a3-4d71-b809-958981901cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:12.682279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61b904-44a3-4d71-b809-958981901cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:12.692124 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61b904-44a3-4d71-b809-958981901cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:12.693543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61b904-44a3-4d71-b809-958981901cb2 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:12.704168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1348/2688] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:03:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:03:13.170614 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6188c-8950-41a6-9014-08b159fa3db8 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:13.172361 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a1e6188c-8950-41a6-9014-08b159fa3db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:13.184505 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6188c-8950-41a6-9014-08b159fa3db8 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:13.190558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941fde2-55cf-4a0f-9005-1bda291e2650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:13.192092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941fde2-55cf-4a0f-9005-1bda291e2650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:13.192383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941fde2-55cf-4a0f-9005-1bda291e2650 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:13.192662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941fde2-55cf-4a0f-9005-1bda291e2650 admin admin] PATH_INFO: `/v3/projects/56a64efcc2324083b99ab916bd2a2ce2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:13.193887 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941fde2-55cf-4a0f-9005-1bda291e2650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56a64efcc2324083b99ab916bd2a2ce2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:13.195839 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941fde2-55cf-4a0f-9005-1bda291e2650 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:13.250260 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-c941fde2-55cf-4a0f-9005-1bda291e2650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56a64efcc2324083b99ab916bd2a2ce2', 'request_id': 'req-c941fde2-55cf-4a0f-9005-1bda291e2650'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:13.257963 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-c941fde2-55cf-4a0f-9005-1bda291e2650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56a64efcc2324083b99ab916bd2a2ce2', 'request_id': 'req-c941fde2-55cf-4a0f-9005-1bda291e2650'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:13.266230 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-c941fde2-55cf-4a0f-9005-1bda291e2650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56a64efcc2324083b99ab916bd2a2ce2', 'request_id': 'req-c941fde2-55cf-4a0f-9005-1bda291e2650'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:13.277666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-c941fde2-55cf-4a0f-9005-1bda291e2650 admin admin] The token cache is being invalidate because project 56a64efcc2324083b99ab916bd2a2ce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:03:13.277952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-c941fde2-55cf-4a0f-9005-1bda291e2650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:13.278587 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-c941fde2-55cf-4a0f-9005-1bda291e2650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:13.280149 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1341/2689] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:03:13 2022] DELETE /identity/v3/projects/56a64efcc2324083b99ab916bd2a2ce2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:03:13.285749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec72b5-b616-4ac2-8c11-e380ab040c11 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:13.285920 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec72b5-b616-4ac2-8c11-e380ab040c11 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:13.286178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec72b5-b616-4ac2-8c11-e380ab040c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:13.305107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-49ec72b5-b616-4ac2-8c11-e380ab040c11 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:03:13.306333 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-49ec72b5-b616-4ac2-8c11-e380ab040c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:13.325206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:03:13.369838 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1349/2690] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:03:13.377575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703e2b4-bbd1-4d66-bf2d-2a0ab698ed7a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:13.377875 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703e2b4-bbd1-4d66-bf2d-2a0ab698ed7a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:13.378158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703e2b4-bbd1-4d66-bf2d-2a0ab698ed7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:13.398116 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-9703e2b4-bbd1-4d66-bf2d-2a0ab698ed7a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:03:13.400005 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9703e2b4-bbd1-4d66-bf2d-2a0ab698ed7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:13.410578 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1342/2691] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:03:13.414754 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec72b5-b616-4ac2-8c11-e380ab040c11 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:13.425309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec72b5-b616-4ac2-8c11-e380ab040c11 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:13.430214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a81882-cf56-4287-9a31-de583e6ba8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:13.430881 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a81882-cf56-4287-9a31-de583e6ba8ff admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:13.431180 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a81882-cf56-4287-9a31-de583e6ba8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:13.431455 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a81882-cf56-4287-9a31-de583e6ba8ff admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:13.432183 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a81882-cf56-4287-9a31-de583e6ba8ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:03:13.432487 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a81882-cf56-4287-9a31-de583e6ba8ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:13.434104 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a81882-cf56-4287-9a31-de583e6ba8ff admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:13.445906 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1350/2692] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:03:13.450196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703e2b4-bbd1-4d66-bf2d-2a0ab698ed7a None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:13.459646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703e2b4-bbd1-4d66-bf2d-2a0ab698ed7a None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:13.464357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ff14c-86e3-4e58-8098-05b2926e8a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:13.465011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ff14c-86e3-4e58-8098-05b2926e8a81 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:13.465305 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ff14c-86e3-4e58-8098-05b2926e8a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:13.465573 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ff14c-86e3-4e58-8098-05b2926e8a81 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:13.466263 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ff14c-86e3-4e58-8098-05b2926e8a81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:13.468234 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ff14c-86e3-4e58-8098-05b2926e8a81 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:13.488534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:03:13.496738 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1343/2693] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:03:13 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:03:13.500562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a81882-cf56-4287-9a31-de583e6ba8ff admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:13.510603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a81882-cf56-4287-9a31-de583e6ba8ff admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:13.566979 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd6901-a057-44f3-a4ad-e3d02537dd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:13.567655 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd6901-a057-44f3-a4ad-e3d02537dd06 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:13.567939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd6901-a057-44f3-a4ad-e3d02537dd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:13.568219 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd6901-a057-44f3-a4ad-e3d02537dd06 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:13.568925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd6901-a057-44f3-a4ad-e3d02537dd06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:13.570852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd6901-a057-44f3-a4ad-e3d02537dd06 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:13.594373 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1351/2694] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:03:13 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:03:13.598458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ff14c-86e3-4e58-8098-05b2926e8a81 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:13.608650 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ff14c-86e3-4e58-8098-05b2926e8a81 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:13.613995 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb92fa-2948-4693-9e2a-69c1500e7939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:13.614800 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb92fa-2948-4693-9e2a-69c1500e7939 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:13.615186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb92fa-2948-4693-9e2a-69c1500e7939 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:13.615533 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb92fa-2948-4693-9e2a-69c1500e7939 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:13.616354 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb92fa-2948-4693-9e2a-69c1500e7939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:13.618013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb92fa-2948-4693-9e2a-69c1500e7939 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:13.628863 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1344/2695] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:03:13.633389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd6901-a057-44f3-a4ad-e3d02537dd06 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:13.644086 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd6901-a057-44f3-a4ad-e3d02537dd06 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:13.648712 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2766801-bc77-4467-8a04-0e164e100174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:13.649434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2766801-bc77-4467-8a04-0e164e100174 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:13.649736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2766801-bc77-4467-8a04-0e164e100174 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:13.650174 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2766801-bc77-4467-8a04-0e164e100174 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:13.651024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2766801-bc77-4467-8a04-0e164e100174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:13.652648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2766801-bc77-4467-8a04-0e164e100174 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:13.666924 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1352/2696] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:03:13.668353 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb92fa-2948-4693-9e2a-69c1500e7939 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:13.678395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb92fa-2948-4693-9e2a-69c1500e7939 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:13.683006 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e1d43-606b-48f3-a27c-156a33286e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:13.683882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e1d43-606b-48f3-a27c-156a33286e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:13.684165 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e1d43-606b-48f3-a27c-156a33286e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:13.684437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e1d43-606b-48f3-a27c-156a33286e1a admin admin] PATH_INFO: `/v3/projects/8f658c67bd7245eaaadf3f969fc562b5/users/5db2bab7613543bbab2db936926966b3/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:13.700147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e1d43-606b-48f3-a27c-156a33286e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f658c67bd7245eaaadf3f969fc562b5, user_id=5db2bab7613543bbab2db936926966b3, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:13.703449 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e1d43-606b-48f3-a27c-156a33286e1a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:13.721272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1345/2697] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:03:13 2022] PUT /identity/v3/projects/8f658c67bd7245eaaadf3f969fc562b5/users/5db2bab7613543bbab2db936926966b3/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:03:13.732586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b874f1-ff3a-4ebc-ac02-31dc4ec4cea8 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:13.732884 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b874f1-ff3a-4ebc-ac02-31dc4ec4cea8 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:13.733218 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b874f1-ff3a-4ebc-ac02-31dc4ec4cea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:13.761937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-85b874f1-ff3a-4ebc-ac02-31dc4ec4cea8 None None] MFA Rules not processed for user `5db2bab7613543bbab2db936926966b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:03:13.763158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-85b874f1-ff3a-4ebc-ac02-31dc4ec4cea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:13.831438 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1353/2698] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:03:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:03:13.842066 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e1d43-606b-48f3-a27c-156a33286e1a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:13.844606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f76e1d43-606b-48f3-a27c-156a33286e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:13.904400 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e1d43-606b-48f3-a27c-156a33286e1a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:13.908577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2c175-8a38-4480-b475-445b4d578d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:13.909220 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c175-8a38-4480-b475-445b4d578d6b service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:13.909506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c175-8a38-4480-b475-445b4d578d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:13.909785 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c175-8a38-4480-b475-445b4d578d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:13.918317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2c175-8a38-4480-b475-445b4d578d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:13.919772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2c175-8a38-4480-b475-445b4d578d6b service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:13.929372 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1346/2699] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:03:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:03:15.364861 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005efcdd-c974-4595-8d74-9cafcd4e47fd None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:15.365464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005efcdd-c974-4595-8d74-9cafcd4e47fd None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:15.365464 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005efcdd-c974-4595-8d74-9cafcd4e47fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:15.384005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-005efcdd-c974-4595-8d74-9cafcd4e47fd None None] MFA Rules not processed for user `5db2bab7613543bbab2db936926966b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:03:15.388814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-005efcdd-c974-4595-8d74-9cafcd4e47fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:15.392726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1354/2700] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:03:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:03:15.990341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7f54f-d559-4ac2-843b-605eed3b82c4 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:15.990813 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7f54f-d559-4ac2-843b-605eed3b82c4 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:15.990998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7f54f-d559-4ac2-843b-605eed3b82c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:16.008993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-56e7f54f-d559-4ac2-843b-605eed3b82c4 None None] MFA Rules not processed for user `df54f67274d24463961e1f48dcb37447`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:03:16.010494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-56e7f54f-d559-4ac2-843b-605eed3b82c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:16.077742 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1347/2701] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:03:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:03:25.378267 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:03:26.070799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=66,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:03:36.598501 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd1784-09e0-4672-b17b-2cdc8b373a66 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:36.599019 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd1784-09e0-4672-b17b-2cdc8b373a66 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:36.599132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd1784-09e0-4672-b17b-2cdc8b373a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:36.616443 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-a9bd1784-09e0-4672-b17b-2cdc8b373a66 None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:03:36.617637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a9bd1784-09e0-4672-b17b-2cdc8b373a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:36.671695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1355/2702] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 14:03:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:03:36.716435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7f54f-d559-4ac2-843b-605eed3b82c4 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:36.717828 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-56e7f54f-d559-4ac2-843b-605eed3b82c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:36.787068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7f54f-d559-4ac2-843b-605eed3b82c4 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:36.792159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be5851-8b1b-44ee-8cc0-6768841912c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:36.792811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be5851-8b1b-44ee-8cc0-6768841912c2 service swift] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:36.793098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be5851-8b1b-44ee-8cc0-6768841912c2 service swift] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:36.793375 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be5851-8b1b-44ee-8cc0-6768841912c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:36.803567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be5851-8b1b-44ee-8cc0-6768841912c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:36.805219 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be5851-8b1b-44ee-8cc0-6768841912c2 service swift] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:36.814754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1348/2703] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 14:03:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:03:37.332395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e1d0e-ded9-475a-9c95-df6484322fe5 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:37.332694 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e1d0e-ded9-475a-9c95-df6484322fe5 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:37.332986 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e1d0e-ded9-475a-9c95-df6484322fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:37.351187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-723e1d0e-ded9-475a-9c95-df6484322fe5 None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:03:37.352387 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-723e1d0e-ded9-475a-9c95-df6484322fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:37.359593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1356/2704] 10.222.0.52 () {58 vars in 967 bytes} [Thu May 19 14:03:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:03:37.372189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be5851-8b1b-44ee-8cc0-6768841912c2 service swift] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:37.383487 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be5851-8b1b-44ee-8cc0-6768841912c2 service swift] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:37.388630 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2ab8c-56d6-4271-8e62-2b6660c7374a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:37.389268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2ab8c-56d6-4271-8e62-2b6660c7374a service swift] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:37.389553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2ab8c-56d6-4271-8e62-2b6660c7374a service swift] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:37.389833 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2ab8c-56d6-4271-8e62-2b6660c7374a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:37.398644 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2ab8c-56d6-4271-8e62-2b6660c7374a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:37.400393 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2ab8c-56d6-4271-8e62-2b6660c7374a service swift] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:37.410390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1349/2705] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 14:03:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:03:37.602285 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356859b4-cf5a-4247-98e2-99818b7ddf68 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:37.602771 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356859b4-cf5a-4247-98e2-99818b7ddf68 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:37.602901 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356859b4-cf5a-4247-98e2-99818b7ddf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:37.620671 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-356859b4-cf5a-4247-98e2-99818b7ddf68 None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:03:37.621856 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-356859b4-cf5a-4247-98e2-99818b7ddf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:37.629648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1357/2706] 10.222.0.52 () {58 vars in 1009 bytes} [Thu May 19 14:03:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:03:37.641672 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2ab8c-56d6-4271-8e62-2b6660c7374a service swift] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:37.652253 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2ab8c-56d6-4271-8e62-2b6660c7374a service swift] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:37.657683 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df07fd-9899-4ddf-88c8-ffd06d7d780a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:37.658328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df07fd-9899-4ddf-88c8-ffd06d7d780a service swift] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:37.658616 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df07fd-9899-4ddf-88c8-ffd06d7d780a service swift] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:37.658922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df07fd-9899-4ddf-88c8-ffd06d7d780a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:37.667446 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df07fd-9899-4ddf-88c8-ffd06d7d780a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:37.669020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df07fd-9899-4ddf-88c8-ffd06d7d780a service swift] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:37.677784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1350/2707] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 14:03:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:03:38.067160 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d01476-84af-48e4-9ff7-7e73fbe9d8a6 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:38.067641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d01476-84af-48e4-9ff7-7e73fbe9d8a6 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:38.068030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d01476-84af-48e4-9ff7-7e73fbe9d8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:38.086955 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-34d01476-84af-48e4-9ff7-7e73fbe9d8a6 None None] MFA Rules not processed for user `a989e5551c1640ebbde8768247441627`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:03:38.088435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-34d01476-84af-48e4-9ff7-7e73fbe9d8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:38.098348 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1358/2708] 10.222.0.52 () {58 vars in 967 bytes} [Thu May 19 14:03:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:03:38.111497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df07fd-9899-4ddf-88c8-ffd06d7d780a service swift] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:38.122985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df07fd-9899-4ddf-88c8-ffd06d7d780a service swift] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:38.128810 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3804d-deb8-42f3-aa23-7637bb56fd6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8554ff40bbcb4fbb801d854d6527a6d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:38.129537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3804d-deb8-42f3-aa23-7637bb56fd6e service swift] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:38.129890 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3804d-deb8-42f3-aa23-7637bb56fd6e service swift] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:38.130238 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3804d-deb8-42f3-aa23-7637bb56fd6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:38.140316 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3804d-deb8-42f3-aa23-7637bb56fd6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:38.142937 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3804d-deb8-42f3-aa23-7637bb56fd6e service swift] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:38.153772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1351/2709] 10.222.0.52 () {60 vars in 1384 bytes} [Thu May 19 14:03:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 14:03:40.255342 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d01476-84af-48e4-9ff7-7e73fbe9d8a6 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:40.320953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d01476-84af-48e4-9ff7-7e73fbe9d8a6 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:40.325568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:40.326434 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:40.326722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:40.327036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] PATH_INFO: `/v3/users/5db2bab7613543bbab2db936926966b3` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:40.328263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5db2bab7613543bbab2db936926966b3)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:40.331291 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:40.379327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5db2bab7613543bbab2db936926966b3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:40.388937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5db2bab7613543bbab2db936926966b3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:40.401802 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5db2bab7613543bbab2db936926966b3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:40.402659 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5db2bab7613543bbab2db936926966b3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:40.403369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5db2bab7613543bbab2db936926966b3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:40.424664 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1359/2710] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:03:40 2022] DELETE /identity/v3/users/5db2bab7613543bbab2db936926966b3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:03:40.429174 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c83b7a-b8e3-4ead-88ed-c80fcc86694a None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:40.429652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c83b7a-b8e3-4ead-88ed-c80fcc86694a None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:40.429753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c83b7a-b8e3-4ead-88ed-c80fcc86694a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:40.450847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-41c83b7a-b8e3-4ead-88ed-c80fcc86694a None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:03:40.452551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-41c83b7a-b8e3-4ead-88ed-c80fcc86694a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:40.527246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1352/2711] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:03:40.537036 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:40.538603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:40.549726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:03:40.616842 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3904cc2-f399-40cf-aede-fa72f273f6a6 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:40.621310 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8485093-5771-4a36-8afd-c42d3b84849a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:40.622691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8485093-5771-4a36-8afd-c42d3b84849a service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:40.623283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8485093-5771-4a36-8afd-c42d3b84849a service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:40.623439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8485093-5771-4a36-8afd-c42d3b84849a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:40.636941 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8485093-5771-4a36-8afd-c42d3b84849a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:40.638394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8485093-5771-4a36-8afd-c42d3b84849a service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:40.649829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1360/2712] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:03:40.768029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c83b7a-b8e3-4ead-88ed-c80fcc86694a None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:40.784709 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c83b7a-b8e3-4ead-88ed-c80fcc86694a None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:40.789927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06006f8e-934f-4233-a2c6-0777bed0a883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:40.791722 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06006f8e-934f-4233-a2c6-0777bed0a883 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:40.792765 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06006f8e-934f-4233-a2c6-0777bed0a883 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:40.793489 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06006f8e-934f-4233-a2c6-0777bed0a883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:40.795930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-06006f8e-934f-4233-a2c6-0777bed0a883 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:40.857529 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=100 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:03:40.873055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8485093-5771-4a36-8afd-c42d3b84849a service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:03:40.874623 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f8485093-5771-4a36-8afd-c42d3b84849a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:03:40.875619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06006f8e-934f-4233-a2c6-0777bed0a883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:40.877740 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06006f8e-934f-4233-a2c6-0777bed0a883 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:40.888384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1353/2713] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:03:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:03:40.901572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8485093-5771-4a36-8afd-c42d3b84849a service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:03:40.915128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:03:40.916014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:03:40.916299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:03:40.917205 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] PATH_INFO: `/v3/projects/8f658c67bd7245eaaadf3f969fc562b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:03:40.918466 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f658c67bd7245eaaadf3f969fc562b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:03:40.920424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:03:40.976718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f658c67bd7245eaaadf3f969fc562b5', 'request_id': 'req-0f3317e4-462c-486c-9c04-686fed1d4767'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:40.982570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f658c67bd7245eaaadf3f969fc562b5', 'request_id': 'req-0f3317e4-462c-486c-9c04-686fed1d4767'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:40.990676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f658c67bd7245eaaadf3f969fc562b5', 'request_id': 'req-0f3317e4-462c-486c-9c04-686fed1d4767'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:41.002489 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] The token cache is being invalidate because project 8f658c67bd7245eaaadf3f969fc562b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:03:41.002711 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:41.003411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:03:41.005155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1361/2714] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:03:40 2022] DELETE /identity/v3/projects/8f658c67bd7245eaaadf3f969fc562b5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:03:50.884208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=7 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:03:50.998764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:02.095396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06006f8e-934f-4233-a2c6-0777bed0a883 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:02.097874 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-06006f8e-934f-4233-a2c6-0777bed0a883 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:02.112185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06006f8e-934f-4233-a2c6-0777bed0a883 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:02.167196 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:02.168062 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:02.168339 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:02.168608 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] PATH_INFO: `/v3/users/12d3af57f5e34f798336d353a111cefd` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:02.169780 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d3af57f5e34f798336d353a111cefd)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:02.171712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:02.218544 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d3af57f5e34f798336d353a111cefd'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:02.225464 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d3af57f5e34f798336d353a111cefd'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:02.226127 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d3af57f5e34f798336d353a111cefd'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:02.232929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d3af57f5e34f798336d353a111cefd'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:02.246137 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d3af57f5e34f798336d353a111cefd'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:02.253758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1354/2715] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:04:02 2022] DELETE /identity/v3/users/12d3af57f5e34f798336d353a111cefd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:02.421114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:02.422568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:02.438337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3317e4-462c-486c-9c04-686fed1d4767 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:02.499524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:02.500287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:02.500566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:02.500857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] PATH_INFO: `/v3/projects/f68358e9d695445490f8209d832a1c13` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:02.502119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f68358e9d695445490f8209d832a1c13)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:02.504191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:02.550311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f68358e9d695445490f8209d832a1c13', 'request_id': 'req-2f01e425-55ca-4626-a1d4-59889443f12c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:02.555675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f68358e9d695445490f8209d832a1c13', 'request_id': 'req-2f01e425-55ca-4626-a1d4-59889443f12c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:02.563078 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f68358e9d695445490f8209d832a1c13', 'request_id': 'req-2f01e425-55ca-4626-a1d4-59889443f12c'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:02.575173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] The token cache is being invalidate because project f68358e9d695445490f8209d832a1c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:04:02.575470 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:02.576108 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:02.577644 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1362/2716] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:04:02 2022] DELETE /identity/v3/projects/f68358e9d695445490f8209d832a1c13 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:12.249228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:12.572047 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:21.212940 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:21.214362 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:21.271061 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b78b6-f999-4636-a1e9-3e2346db9cff admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:21.318197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb260350-0981-4018-a6cb-a11960537735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:21.318837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb260350-0981-4018-a6cb-a11960537735 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:21.319151 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb260350-0981-4018-a6cb-a11960537735 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:21.319425 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb260350-0981-4018-a6cb-a11960537735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:21.321200 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-cb260350-0981-4018-a6cb-a11960537735 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:21.390374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb260350-0981-4018-a6cb-a11960537735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:21.391853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb260350-0981-4018-a6cb-a11960537735 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:21.403271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1355/2717] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:04:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:04:25.649834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:25.651398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:25.710494 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01e425-55ca-4626-a1d4-59889443f12c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:25.714135 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b7cbc-5d44-4faa-ac31-a1407f648bc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:25.714784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b7cbc-5d44-4faa-ac31-a1407f648bc7 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:25.715104 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b7cbc-5d44-4faa-ac31-a1407f648bc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:25.715386 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b7cbc-5d44-4faa-ac31-a1407f648bc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:25.717195 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-459b7cbc-5d44-4faa-ac31-a1407f648bc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:25.732133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b7cbc-5d44-4faa-ac31-a1407f648bc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:25.733570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b7cbc-5d44-4faa-ac31-a1407f648bc7 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:25.745396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1363/2718] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:04:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:04:31.398909 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=74 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:35.356049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb260350-0981-4018-a6cb-a11960537735 service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:35.359010 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-cb260350-0981-4018-a6cb-a11960537735 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:35.457657 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb260350-0981-4018-a6cb-a11960537735 service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:35.463436 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877b3c6-8223-40c4-b4ff-2ce44a803d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:35.464518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877b3c6-8223-40c4-b4ff-2ce44a803d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:35.464842 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877b3c6-8223-40c4-b4ff-2ce44a803d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:35.465164 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877b3c6-8223-40c4-b4ff-2ce44a803d78 admin admin] PATH_INFO: `/v3/users/ab1b34a915464792b5d3b74956bc49a2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:35.466652 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877b3c6-8223-40c4-b4ff-2ce44a803d78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab1b34a915464792b5d3b74956bc49a2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:35.469044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877b3c6-8223-40c4-b4ff-2ce44a803d78 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:35.544729 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6877b3c6-8223-40c4-b4ff-2ce44a803d78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1b34a915464792b5d3b74956bc49a2'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:35.553462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6877b3c6-8223-40c4-b4ff-2ce44a803d78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1b34a915464792b5d3b74956bc49a2'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:35.554120 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6877b3c6-8223-40c4-b4ff-2ce44a803d78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1b34a915464792b5d3b74956bc49a2'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:35.568048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6877b3c6-8223-40c4-b4ff-2ce44a803d78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1b34a915464792b5d3b74956bc49a2'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:35.589644 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6877b3c6-8223-40c4-b4ff-2ce44a803d78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1b34a915464792b5d3b74956bc49a2'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:35.598997 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1356/2719] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:04:35 2022] DELETE /identity/v3/users/ab1b34a915464792b5d3b74956bc49a2 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:35.740063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=26 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:35.796311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b7cbc-5d44-4faa-ac31-a1407f648bc7 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:35.808038 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b7cbc-5d44-4faa-ac31-a1407f648bc7 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:35.886062 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785724b-1e03-485f-943b-0909ce654ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:35.886972 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785724b-1e03-485f-943b-0909ce654ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:35.887284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785724b-1e03-485f-943b-0909ce654ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:35.887561 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785724b-1e03-485f-943b-0909ce654ab7 admin admin] PATH_INFO: `/v3/projects/e14dac6dd56c4f93b65ad91673debf71` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:35.888847 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785724b-1e03-485f-943b-0909ce654ab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e14dac6dd56c4f93b65ad91673debf71)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:35.890779 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785724b-1e03-485f-943b-0909ce654ab7 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:35.942673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0785724b-1e03-485f-943b-0909ce654ab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e14dac6dd56c4f93b65ad91673debf71', 'request_id': 'req-0785724b-1e03-485f-943b-0909ce654ab7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:35.948154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0785724b-1e03-485f-943b-0909ce654ab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e14dac6dd56c4f93b65ad91673debf71', 'request_id': 'req-0785724b-1e03-485f-943b-0909ce654ab7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:35.958991 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0785724b-1e03-485f-943b-0909ce654ab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e14dac6dd56c4f93b65ad91673debf71', 'request_id': 'req-0785724b-1e03-485f-943b-0909ce654ab7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:35.970900 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0785724b-1e03-485f-943b-0909ce654ab7 admin admin] The token cache is being invalidate because project e14dac6dd56c4f93b65ad91673debf71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:04:35.971191 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0785724b-1e03-485f-943b-0909ce654ab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:35.971846 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0785724b-1e03-485f-943b-0909ce654ab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:35.973440 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1364/2720] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:04:35 2022] DELETE /identity/v3/projects/e14dac6dd56c4f93b65ad91673debf71 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:04:35.978395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832c8bb-c671-428a-81a0-cd31a72d1045 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:35.978679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832c8bb-c671-428a-81a0-cd31a72d1045 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:35.978986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832c8bb-c671-428a-81a0-cd31a72d1045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:36.000668 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-8832c8bb-c671-428a-81a0-cd31a72d1045 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:36.001900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-8832c8bb-c671-428a-81a0-cd31a72d1045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:36.069323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1357/2721] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:04:36.077725 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95019e1a-20f4-40d1-aea8-7711d8baba56 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:36.078065 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95019e1a-20f4-40d1-aea8-7711d8baba56 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:36.078394 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95019e1a-20f4-40d1-aea8-7711d8baba56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:36.098685 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-95019e1a-20f4-40d1-aea8-7711d8baba56 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:36.099953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-95019e1a-20f4-40d1-aea8-7711d8baba56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:36.108526 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1365/2722] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:04:36.113078 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832c8bb-c671-428a-81a0-cd31a72d1045 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:36.123410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832c8bb-c671-428a-81a0-cd31a72d1045 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:36.129001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38691c0-4f86-4388-82f2-05a2a17408d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:36.129678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38691c0-4f86-4388-82f2-05a2a17408d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:36.129962 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38691c0-4f86-4388-82f2-05a2a17408d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:36.130241 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38691c0-4f86-4388-82f2-05a2a17408d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:36.130996 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38691c0-4f86-4388-82f2-05a2a17408d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:04:36.131306 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38691c0-4f86-4388-82f2-05a2a17408d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:36.133203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38691c0-4f86-4388-82f2-05a2a17408d3 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:36.146662 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1358/2723] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:04:36.151350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95019e1a-20f4-40d1-aea8-7711d8baba56 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:36.161231 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95019e1a-20f4-40d1-aea8-7711d8baba56 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:36.167102 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e846a-e98c-405a-826a-0d2516333036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:36.167793 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e846a-e98c-405a-826a-0d2516333036 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:36.168078 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e846a-e98c-405a-826a-0d2516333036 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:36.168355 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e846a-e98c-405a-826a-0d2516333036 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:36.169124 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e846a-e98c-405a-826a-0d2516333036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:36.171096 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e846a-e98c-405a-826a-0d2516333036 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:36.199615 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1366/2724] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:04:36 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:04:36.204121 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38691c0-4f86-4388-82f2-05a2a17408d3 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:36.215585 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38691c0-4f86-4388-82f2-05a2a17408d3 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:36.219606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:36.291855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13f712-737f-4754-b274-51a790ce5036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:36.292697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13f712-737f-4754-b274-51a790ce5036 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:36.292988 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13f712-737f-4754-b274-51a790ce5036 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:36.293273 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13f712-737f-4754-b274-51a790ce5036 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:36.293972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13f712-737f-4754-b274-51a790ce5036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:36.296303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13f712-737f-4754-b274-51a790ce5036 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:36.321090 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1359/2725] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:04:36 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:04:36.327268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e846a-e98c-405a-826a-0d2516333036 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:36.339175 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e846a-e98c-405a-826a-0d2516333036 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:36.344035 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ea6b7-227d-4da1-b97a-00af8c7c6d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:36.344731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ea6b7-227d-4da1-b97a-00af8c7c6d9c admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:36.345030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ea6b7-227d-4da1-b97a-00af8c7c6d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:36.345295 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ea6b7-227d-4da1-b97a-00af8c7c6d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:36.346032 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ea6b7-227d-4da1-b97a-00af8c7c6d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:36.348284 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ea6b7-227d-4da1-b97a-00af8c7c6d9c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:36.359750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1367/2726] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:04:36.363974 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13f712-737f-4754-b274-51a790ce5036 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:36.379857 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13f712-737f-4754-b274-51a790ce5036 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:36.385463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42271c-1c7b-460b-a81f-c7af3bdba0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:36.386159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42271c-1c7b-460b-a81f-c7af3bdba0dc admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:36.386452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42271c-1c7b-460b-a81f-c7af3bdba0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:36.386718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42271c-1c7b-460b-a81f-c7af3bdba0dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:36.387509 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42271c-1c7b-460b-a81f-c7af3bdba0dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:36.389120 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42271c-1c7b-460b-a81f-c7af3bdba0dc admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:36.404834 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1360/2727] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:04:36.408987 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ea6b7-227d-4da1-b97a-00af8c7c6d9c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:36.420893 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ea6b7-227d-4da1-b97a-00af8c7c6d9c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:36.431347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdefd1-4161-4e6d-a82a-0005ad31592e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:36.432836 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdefd1-4161-4e6d-a82a-0005ad31592e admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:36.433302 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdefd1-4161-4e6d-a82a-0005ad31592e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:36.433609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdefd1-4161-4e6d-a82a-0005ad31592e admin admin] PATH_INFO: `/v3/projects/6c28119e5ebe4d5d99cf71063c21af1d/users/279fba51d41e4547915ea4cf2cd8b78b/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:36.449725 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdefd1-4161-4e6d-a82a-0005ad31592e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c28119e5ebe4d5d99cf71063c21af1d, user_id=279fba51d41e4547915ea4cf2cd8b78b, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:36.452438 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdefd1-4161-4e6d-a82a-0005ad31592e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:36.468141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1368/2728] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:04:36 2022] PUT /identity/v3/projects/6c28119e5ebe4d5d99cf71063c21af1d/users/279fba51d41e4547915ea4cf2cd8b78b/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:36.480502 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d735f-fbbe-4a1c-8822-26fca258ae1d None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:36.480790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d735f-fbbe-4a1c-8822-26fca258ae1d None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:36.481073 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d735f-fbbe-4a1c-8822-26fca258ae1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:36.511029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-397d735f-fbbe-4a1c-8822-26fca258ae1d None None] MFA Rules not processed for user `279fba51d41e4547915ea4cf2cd8b78b`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:36.512242 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-397d735f-fbbe-4a1c-8822-26fca258ae1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:36.576358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1361/2729] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:04:36.587862 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdefd1-4161-4e6d-a82a-0005ad31592e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:36.589592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-71bdefd1-4161-4e6d-a82a-0005ad31592e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:36.623118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:36.670256 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdefd1-4161-4e6d-a82a-0005ad31592e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:36.676528 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ad916-0b9e-46a5-b07f-76ee9fbf3b20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:36.677184 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ad916-0b9e-46a5-b07f-76ee9fbf3b20 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:36.677474 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ad916-0b9e-46a5-b07f-76ee9fbf3b20 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:36.677761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ad916-0b9e-46a5-b07f-76ee9fbf3b20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:36.686299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ad916-0b9e-46a5-b07f-76ee9fbf3b20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:36.687930 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ad916-0b9e-46a5-b07f-76ee9fbf3b20 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:36.698475 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1369/2730] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:04:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:04:39.366371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d735f-fbbe-4a1c-8822-26fca258ae1d None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:39.430583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d735f-fbbe-4a1c-8822-26fca258ae1d None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:39.435250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:39.436126 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:39.436410 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:39.436688 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] PATH_INFO: `/v3/users/279fba51d41e4547915ea4cf2cd8b78b` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:39.437889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=279fba51d41e4547915ea4cf2cd8b78b)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:39.439871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:39.456476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:39.486326 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '279fba51d41e4547915ea4cf2cd8b78b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:39.493757 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '279fba51d41e4547915ea4cf2cd8b78b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:39.494567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '279fba51d41e4547915ea4cf2cd8b78b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:39.501902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '279fba51d41e4547915ea4cf2cd8b78b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:39.514918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '279fba51d41e4547915ea4cf2cd8b78b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:39.521235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1362/2731] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:04:39 2022] DELETE /identity/v3/users/279fba51d41e4547915ea4cf2cd8b78b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:39.525756 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4f7d2-bc5d-453b-8bb4-e963b38c2fb7 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:39.526224 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4f7d2-bc5d-453b-8bb4-e963b38c2fb7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:39.526354 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4f7d2-bc5d-453b-8bb4-e963b38c2fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:39.543937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-a5b4f7d2-bc5d-453b-8bb4-e963b38c2fb7 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:39.545140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a5b4f7d2-bc5d-453b-8bb4-e963b38c2fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:39.603131 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1370/2732] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:04:39.613591 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:39.615050 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:39.675704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6576e-a16d-47a4-b2f5-6255b7e4fd02 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:39.679424 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b582c1e-0f39-4ad1-b46e-429549751fa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:39.680076 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582c1e-0f39-4ad1-b46e-429549751fa1 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:39.680363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582c1e-0f39-4ad1-b46e-429549751fa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:39.680637 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582c1e-0f39-4ad1-b46e-429549751fa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:39.689759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b582c1e-0f39-4ad1-b46e-429549751fa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:39.691494 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b582c1e-0f39-4ad1-b46e-429549751fa1 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:39.701728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1363/2733] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:04:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:04:39.982076 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4f7d2-bc5d-453b-8bb4-e963b38c2fb7 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:39.983764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-a5b4f7d2-bc5d-453b-8bb4-e963b38c2fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:39.999559 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4f7d2-bc5d-453b-8bb4-e963b38c2fb7 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:40.005235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed6ad3-fee2-419a-8768-1eef9e5dc412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:40.006317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed6ad3-fee2-419a-8768-1eef9e5dc412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:40.006679 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed6ad3-fee2-419a-8768-1eef9e5dc412 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:40.006988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed6ad3-fee2-419a-8768-1eef9e5dc412 admin admin] PATH_INFO: `/v3/projects/6c28119e5ebe4d5d99cf71063c21af1d` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:40.008459 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed6ad3-fee2-419a-8768-1eef9e5dc412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c28119e5ebe4d5d99cf71063c21af1d)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:40.010683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed6ad3-fee2-419a-8768-1eef9e5dc412 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:40.059317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-22ed6ad3-fee2-419a-8768-1eef9e5dc412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c28119e5ebe4d5d99cf71063c21af1d', 'request_id': 'req-22ed6ad3-fee2-419a-8768-1eef9e5dc412'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:40.064993 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-22ed6ad3-fee2-419a-8768-1eef9e5dc412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c28119e5ebe4d5d99cf71063c21af1d', 'request_id': 'req-22ed6ad3-fee2-419a-8768-1eef9e5dc412'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:40.073684 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-22ed6ad3-fee2-419a-8768-1eef9e5dc412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c28119e5ebe4d5d99cf71063c21af1d', 'request_id': 'req-22ed6ad3-fee2-419a-8768-1eef9e5dc412'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:40.085707 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-22ed6ad3-fee2-419a-8768-1eef9e5dc412 admin admin] The token cache is being invalidate because project 6c28119e5ebe4d5d99cf71063c21af1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:04:40.085993 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-22ed6ad3-fee2-419a-8768-1eef9e5dc412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:40.086637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-22ed6ad3-fee2-419a-8768-1eef9e5dc412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:40.088187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1371/2734] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:04:39 2022] DELETE /identity/v3/projects/6c28119e5ebe4d5d99cf71063c21af1d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:40.094423 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96623e4a-a359-4772-a41f-91b1d6ad5c45 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:40.094712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96623e4a-a359-4772-a41f-91b1d6ad5c45 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:40.095025 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96623e4a-a359-4772-a41f-91b1d6ad5c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:40.113704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-96623e4a-a359-4772-a41f-91b1d6ad5c45 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:40.114964 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-96623e4a-a359-4772-a41f-91b1d6ad5c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:40.188909 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1364/2735] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:04:40.196694 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be5271-9271-4660-9b34-dff2f73893f3 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:40.196987 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be5271-9271-4660-9b34-dff2f73893f3 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:40.197269 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be5271-9271-4660-9b34-dff2f73893f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:40.215916 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-41be5271-9271-4660-9b34-dff2f73893f3 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:40.217112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-41be5271-9271-4660-9b34-dff2f73893f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:40.225608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1372/2736] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:04:40.229882 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96623e4a-a359-4772-a41f-91b1d6ad5c45 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:40.239688 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96623e4a-a359-4772-a41f-91b1d6ad5c45 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:40.244576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fcc3fe-5ebf-4a37-aaed-dd5ded3fa1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:40.245235 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcc3fe-5ebf-4a37-aaed-dd5ded3fa1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:40.245518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcc3fe-5ebf-4a37-aaed-dd5ded3fa1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:40.245793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcc3fe-5ebf-4a37-aaed-dd5ded3fa1a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:40.246518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fcc3fe-5ebf-4a37-aaed-dd5ded3fa1a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:04:40.246950 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fcc3fe-5ebf-4a37-aaed-dd5ded3fa1a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:40.248729 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fcc3fe-5ebf-4a37-aaed-dd5ded3fa1a2 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:40.261607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1365/2737] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:04:40.265937 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be5271-9271-4660-9b34-dff2f73893f3 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:40.276045 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be5271-9271-4660-9b34-dff2f73893f3 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:40.282099 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb9a86-2583-45c5-9887-2c63d3182045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:40.282768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb9a86-2583-45c5-9887-2c63d3182045 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:40.283209 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb9a86-2583-45c5-9887-2c63d3182045 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:40.283489 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb9a86-2583-45c5-9887-2c63d3182045 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:40.285051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb9a86-2583-45c5-9887-2c63d3182045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:40.287150 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb9a86-2583-45c5-9887-2c63d3182045 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:40.314140 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1373/2738] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:04:40 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:04:40.318465 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fcc3fe-5ebf-4a37-aaed-dd5ded3fa1a2 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:40.328719 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fcc3fe-5ebf-4a37-aaed-dd5ded3fa1a2 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:40.350822 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:40.397079 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73113cbf-e597-48ea-96a5-948f1dde510c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:40.397768 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73113cbf-e597-48ea-96a5-948f1dde510c admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:40.398037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73113cbf-e597-48ea-96a5-948f1dde510c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:40.398317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73113cbf-e597-48ea-96a5-948f1dde510c admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:40.399053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73113cbf-e597-48ea-96a5-948f1dde510c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:40.401607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73113cbf-e597-48ea-96a5-948f1dde510c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:40.425985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1366/2739] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:04:40 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:04:40.427327 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb9a86-2583-45c5-9887-2c63d3182045 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:40.437597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb9a86-2583-45c5-9887-2c63d3182045 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:40.443854 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88aa122-70cd-4cc6-8792-c9f999331748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:40.444465 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88aa122-70cd-4cc6-8792-c9f999331748 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:40.444769 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88aa122-70cd-4cc6-8792-c9f999331748 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:40.445014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88aa122-70cd-4cc6-8792-c9f999331748 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:40.445752 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88aa122-70cd-4cc6-8792-c9f999331748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:40.447534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88aa122-70cd-4cc6-8792-c9f999331748 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:40.449010 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:40.462843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1374/2740] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:04:40.466622 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73113cbf-e597-48ea-96a5-948f1dde510c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:40.477171 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73113cbf-e597-48ea-96a5-948f1dde510c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:40.481880 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea4e7c-88b5-407b-a833-d7cc9665de7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:40.482537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4e7c-88b5-407b-a833-d7cc9665de7d admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:40.482824 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4e7c-88b5-407b-a833-d7cc9665de7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:40.483135 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4e7c-88b5-407b-a833-d7cc9665de7d admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:40.483863 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea4e7c-88b5-407b-a833-d7cc9665de7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:40.485433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea4e7c-88b5-407b-a833-d7cc9665de7d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:40.496522 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1367/2741] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:04:40.500925 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88aa122-70cd-4cc6-8792-c9f999331748 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:40.513216 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88aa122-70cd-4cc6-8792-c9f999331748 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:40.518303 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7b351-4ede-4119-9560-3e4c2206323f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:40.520244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7b351-4ede-4119-9560-3e4c2206323f admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:40.520534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7b351-4ede-4119-9560-3e4c2206323f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:40.520820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7b351-4ede-4119-9560-3e4c2206323f admin admin] PATH_INFO: `/v3/projects/9d716fe1ec6b47e6a76c97ac5d4b137b/users/fbcc6c3aaeca4127abdb6ef4445d6ba5/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:40.536429 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7b351-4ede-4119-9560-3e4c2206323f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d716fe1ec6b47e6a76c97ac5d4b137b, user_id=fbcc6c3aaeca4127abdb6ef4445d6ba5, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:40.539112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7b351-4ede-4119-9560-3e4c2206323f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:40.554020 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1375/2742] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:04:40 2022] PUT /identity/v3/projects/9d716fe1ec6b47e6a76c97ac5d4b137b/users/fbcc6c3aaeca4127abdb6ef4445d6ba5/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:40.564242 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb8484-5b3f-40f5-8937-f170e7e8c644 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:40.564540 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb8484-5b3f-40f5-8937-f170e7e8c644 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:40.564824 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb8484-5b3f-40f5-8937-f170e7e8c644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:40.594567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-9bcb8484-5b3f-40f5-8937-f170e7e8c644 None None] MFA Rules not processed for user `fbcc6c3aaeca4127abdb6ef4445d6ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:40.595808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9bcb8484-5b3f-40f5-8937-f170e7e8c644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:40.660601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1368/2743] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:04:40.671319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7b351-4ede-4119-9560-3e4c2206323f admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:40.673000 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-34d7b351-4ede-4119-9560-3e4c2206323f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:40.730543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7b351-4ede-4119-9560-3e4c2206323f admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:40.734353 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8abf3-11b6-4ae2-aa01-b1093cc9d735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:40.735043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8abf3-11b6-4ae2-aa01-b1093cc9d735 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:40.735332 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8abf3-11b6-4ae2-aa01-b1093cc9d735 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:40.735614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8abf3-11b6-4ae2-aa01-b1093cc9d735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:40.745041 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8abf3-11b6-4ae2-aa01-b1093cc9d735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:40.746465 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8abf3-11b6-4ae2-aa01-b1093cc9d735 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:40.756583 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1376/2744] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:04:40 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:04:49.688009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb8484-5b3f-40f5-8937-f170e7e8c644 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:49.762150 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb8484-5b3f-40f5-8937-f170e7e8c644 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:49.767215 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:49.768147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:49.768949 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:49.769286 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] PATH_INFO: `/v3/users/fbcc6c3aaeca4127abdb6ef4445d6ba5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:49.770512 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbcc6c3aaeca4127abdb6ef4445d6ba5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:49.772687 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:49.828943 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:49.843335 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbcc6c3aaeca4127abdb6ef4445d6ba5'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:49.854364 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbcc6c3aaeca4127abdb6ef4445d6ba5'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:49.855564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbcc6c3aaeca4127abdb6ef4445d6ba5'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:49.865802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbcc6c3aaeca4127abdb6ef4445d6ba5'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:49.880583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbcc6c3aaeca4127abdb6ef4445d6ba5'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:49.887432 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1369/2745] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:04:49 2022] DELETE /identity/v3/users/fbcc6c3aaeca4127abdb6ef4445d6ba5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:49.893279 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff81f7-85e0-4731-af5a-f6e076da97e8 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:49.894525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff81f7-85e0-4731-af5a-f6e076da97e8 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:49.894525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff81f7-85e0-4731-af5a-f6e076da97e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:49.915988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-1cff81f7-85e0-4731-af5a-f6e076da97e8 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:49.917265 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1cff81f7-85e0-4731-af5a-f6e076da97e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:49.994564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1377/2746] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:04:50.005624 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:50.007169 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:50.075999 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087b6dc-6cc2-409c-a3d7-b97d2e9ba05d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:50.080136 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a689fb-a338-43a3-802b-a2ac6ce80c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:50.080837 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a689fb-a338-43a3-802b-a2ac6ce80c9a service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:50.081125 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a689fb-a338-43a3-802b-a2ac6ce80c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:50.081444 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a689fb-a338-43a3-802b-a2ac6ce80c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:50.092084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a689fb-a338-43a3-802b-a2ac6ce80c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:50.093689 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a689fb-a338-43a3-802b-a2ac6ce80c9a service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:50.105084 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1370/2747] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:04:50.455059 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff81f7-85e0-4731-af5a-f6e076da97e8 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:50.457144 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1cff81f7-85e0-4731-af5a-f6e076da97e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:50.478219 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff81f7-85e0-4731-af5a-f6e076da97e8 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:50.486018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6028222-cac4-4b00-9f33-34fc20095b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:50.490158 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6028222-cac4-4b00-9f33-34fc20095b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:50.490945 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6028222-cac4-4b00-9f33-34fc20095b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:50.491699 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6028222-cac4-4b00-9f33-34fc20095b9e admin admin] PATH_INFO: `/v3/projects/9d716fe1ec6b47e6a76c97ac5d4b137b` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:50.494439 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6028222-cac4-4b00-9f33-34fc20095b9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d716fe1ec6b47e6a76c97ac5d4b137b)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:50.497511 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6028222-cac4-4b00-9f33-34fc20095b9e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:50.559873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e6028222-cac4-4b00-9f33-34fc20095b9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d716fe1ec6b47e6a76c97ac5d4b137b', 'request_id': 'req-e6028222-cac4-4b00-9f33-34fc20095b9e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:50.566034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e6028222-cac4-4b00-9f33-34fc20095b9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d716fe1ec6b47e6a76c97ac5d4b137b', 'request_id': 'req-e6028222-cac4-4b00-9f33-34fc20095b9e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:50.576572 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e6028222-cac4-4b00-9f33-34fc20095b9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d716fe1ec6b47e6a76c97ac5d4b137b', 'request_id': 'req-e6028222-cac4-4b00-9f33-34fc20095b9e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:50.589098 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:50.595893 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e6028222-cac4-4b00-9f33-34fc20095b9e admin admin] The token cache is being invalidate because project 9d716fe1ec6b47e6a76c97ac5d4b137b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:04:50.597064 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e6028222-cac4-4b00-9f33-34fc20095b9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:50.597749 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-e6028222-cac4-4b00-9f33-34fc20095b9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:50.599574 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1378/2748] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:04:50 2022] DELETE /identity/v3/projects/9d716fe1ec6b47e6a76c97ac5d4b137b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:50.604872 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b70ff-eee8-4623-b236-12f0642bdf6c None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:50.605177 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b70ff-eee8-4623-b236-12f0642bdf6c None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:50.605419 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b70ff-eee8-4623-b236-12f0642bdf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:50.626906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-a34b70ff-eee8-4623-b236-12f0642bdf6c None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:50.628254 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-a34b70ff-eee8-4623-b236-12f0642bdf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:50.697730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1371/2749] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:04:50.707996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9255a4-57ec-4f0e-92d4-8a46f234b73d None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:50.708289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9255a4-57ec-4f0e-92d4-8a46f234b73d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:50.708577 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9255a4-57ec-4f0e-92d4-8a46f234b73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:50.733489 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-1f9255a4-57ec-4f0e-92d4-8a46f234b73d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:50.734876 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1f9255a4-57ec-4f0e-92d4-8a46f234b73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:50.748469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1379/2750] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:04:50.753847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b70ff-eee8-4623-b236-12f0642bdf6c None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:50.766601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b70ff-eee8-4623-b236-12f0642bdf6c None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:50.772052 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67903658-b8d1-40e6-9ab3-a7beb8c9873c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:50.772750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67903658-b8d1-40e6-9ab3-a7beb8c9873c admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:50.773041 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67903658-b8d1-40e6-9ab3-a7beb8c9873c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:50.773321 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67903658-b8d1-40e6-9ab3-a7beb8c9873c admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:50.774082 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67903658-b8d1-40e6-9ab3-a7beb8c9873c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:04:50.774390 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67903658-b8d1-40e6-9ab3-a7beb8c9873c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:50.776137 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67903658-b8d1-40e6-9ab3-a7beb8c9873c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:50.790942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1372/2751] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:04:50.797417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9255a4-57ec-4f0e-92d4-8a46f234b73d None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:50.815113 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9255a4-57ec-4f0e-92d4-8a46f234b73d None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:50.820800 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b132b-442f-4c25-85c6-e398a31d2fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:50.821486 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b132b-442f-4c25-85c6-e398a31d2fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:50.821766 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b132b-442f-4c25-85c6-e398a31d2fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:50.822039 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b132b-442f-4c25-85c6-e398a31d2fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:50.822750 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b132b-442f-4c25-85c6-e398a31d2fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:50.824761 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b132b-442f-4c25-85c6-e398a31d2fb1 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:50.855130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1380/2752] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:04:50 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:04:50.859323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67903658-b8d1-40e6-9ab3-a7beb8c9873c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:50.869961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67903658-b8d1-40e6-9ab3-a7beb8c9873c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:50.919237 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:50.950994 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e52b33-52ce-4f5a-8fa0-51d19927a8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:50.952083 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e52b33-52ce-4f5a-8fa0-51d19927a8fe admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:50.952519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e52b33-52ce-4f5a-8fa0-51d19927a8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:50.952990 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e52b33-52ce-4f5a-8fa0-51d19927a8fe admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:50.954264 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e52b33-52ce-4f5a-8fa0-51d19927a8fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:50.958395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e52b33-52ce-4f5a-8fa0-51d19927a8fe admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:50.986781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1373/2753] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:04:50 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:04:50.991743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b132b-442f-4c25-85c6-e398a31d2fb1 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:51.004521 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b132b-442f-4c25-85c6-e398a31d2fb1 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:51.012196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b3a5d-4a7e-4a2f-8cf3-e69d78f2ce06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:51.013292 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b3a5d-4a7e-4a2f-8cf3-e69d78f2ce06 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:51.013778 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b3a5d-4a7e-4a2f-8cf3-e69d78f2ce06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:51.014260 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b3a5d-4a7e-4a2f-8cf3-e69d78f2ce06 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:51.015657 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b3a5d-4a7e-4a2f-8cf3-e69d78f2ce06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:51.017299 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b3a5d-4a7e-4a2f-8cf3-e69d78f2ce06 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:51.035263 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1381/2754] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:04:51.039402 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e52b33-52ce-4f5a-8fa0-51d19927a8fe admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:51.050407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e52b33-52ce-4f5a-8fa0-51d19927a8fe admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:51.056730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ef69f-400d-4d36-b3dd-4275d6a32cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:51.057692 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ef69f-400d-4d36-b3dd-4275d6a32cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:51.058011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ef69f-400d-4d36-b3dd-4275d6a32cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:51.058289 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ef69f-400d-4d36-b3dd-4275d6a32cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:51.059081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ef69f-400d-4d36-b3dd-4275d6a32cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:51.060770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ef69f-400d-4d36-b3dd-4275d6a32cb1 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:51.077679 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1374/2755] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:04:51.083637 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b3a5d-4a7e-4a2f-8cf3-e69d78f2ce06 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:51.102182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b3a5d-4a7e-4a2f-8cf3-e69d78f2ce06 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:51.107993 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb04ca6-a464-4498-87a6-e40e997d0873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:51.108897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb04ca6-a464-4498-87a6-e40e997d0873 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:51.109250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb04ca6-a464-4498-87a6-e40e997d0873 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:51.109742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb04ca6-a464-4498-87a6-e40e997d0873 admin admin] PATH_INFO: `/v3/projects/d8aa875187e545d996936eed6ff67c1c/users/44d9dd5055224c5b8e11dba387dd98ed/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:51.137347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb04ca6-a464-4498-87a6-e40e997d0873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8aa875187e545d996936eed6ff67c1c, user_id=44d9dd5055224c5b8e11dba387dd98ed, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:51.140040 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb04ca6-a464-4498-87a6-e40e997d0873 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:51.155102 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1382/2756] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:04:51 2022] PUT /identity/v3/projects/d8aa875187e545d996936eed6ff67c1c/users/44d9dd5055224c5b8e11dba387dd98ed/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:51.166770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee7a81-341c-4b6c-9b0a-527ba50ec335 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:51.167068 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee7a81-341c-4b6c-9b0a-527ba50ec335 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:51.167357 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee7a81-341c-4b6c-9b0a-527ba50ec335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:51.206728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-00ee7a81-341c-4b6c-9b0a-527ba50ec335 None None] MFA Rules not processed for user `44d9dd5055224c5b8e11dba387dd98ed`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:51.208194 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-00ee7a81-341c-4b6c-9b0a-527ba50ec335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:51.283430 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1375/2757] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:04:51.288875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb04ca6-a464-4498-87a6-e40e997d0873 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:51.367162 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb04ca6-a464-4498-87a6-e40e997d0873 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:51.372006 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c1ad1-858d-4f06-ba5b-b9f6b3b5b8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:51.372671 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c1ad1-858d-4f06-ba5b-b9f6b3b5b8ff admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:51.372967 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c1ad1-858d-4f06-ba5b-b9f6b3b5b8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:51.373243 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c1ad1-858d-4f06-ba5b-b9f6b3b5b8ff admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:51.373964 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c1ad1-858d-4f06-ba5b-b9f6b3b5b8ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:51.375927 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c1ad1-858d-4f06-ba5b-b9f6b3b5b8ff admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:51.408029 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1383/2758] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:04:51 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:04:51.413955 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee7a81-341c-4b6c-9b0a-527ba50ec335 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:51.430388 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee7a81-341c-4b6c-9b0a-527ba50ec335 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:51.506350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffcff0-90f2-4f1a-8a0d-b571692bd322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:51.507085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffcff0-90f2-4f1a-8a0d-b571692bd322 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:51.507511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffcff0-90f2-4f1a-8a0d-b571692bd322 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:51.508203 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffcff0-90f2-4f1a-8a0d-b571692bd322 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:51.509437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffcff0-90f2-4f1a-8a0d-b571692bd322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:51.512611 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffcff0-90f2-4f1a-8a0d-b571692bd322 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:51.545685 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1376/2759] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:04:51 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:04:51.552054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c1ad1-858d-4f06-ba5b-b9f6b3b5b8ff admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:51.570097 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c1ad1-858d-4f06-ba5b-b9f6b3b5b8ff admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:51.575092 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01aa09-1747-41a9-8eb8-eba9222ef3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:51.575766 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01aa09-1747-41a9-8eb8-eba9222ef3ac admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:51.576056 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01aa09-1747-41a9-8eb8-eba9222ef3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:51.576341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01aa09-1747-41a9-8eb8-eba9222ef3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:51.577286 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01aa09-1747-41a9-8eb8-eba9222ef3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:51.580193 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01aa09-1747-41a9-8eb8-eba9222ef3ac admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:51.601173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1384/2760] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:04:51.606295 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffcff0-90f2-4f1a-8a0d-b571692bd322 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:51.625023 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffcff0-90f2-4f1a-8a0d-b571692bd322 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:51.634644 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d31cc-b285-435d-a85d-02518ef585dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:51.636281 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d31cc-b285-435d-a85d-02518ef585dd admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:51.636799 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d31cc-b285-435d-a85d-02518ef585dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:51.637292 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d31cc-b285-435d-a85d-02518ef585dd admin admin] PATH_INFO: `/v3/projects/1706fc562cb44a5cad5cfab887fe1179/users/ce6a062bc8fc4915b70256b47bd9d5c1/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:51.665461 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d31cc-b285-435d-a85d-02518ef585dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1706fc562cb44a5cad5cfab887fe1179, user_id=ce6a062bc8fc4915b70256b47bd9d5c1, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:51.670108 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d31cc-b285-435d-a85d-02518ef585dd admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:51.688440 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1377/2761] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:04:51 2022] PUT /identity/v3/projects/1706fc562cb44a5cad5cfab887fe1179/users/ce6a062bc8fc4915b70256b47bd9d5c1/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:51.694484 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01aa09-1747-41a9-8eb8-eba9222ef3ac admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:51.706888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01aa09-1747-41a9-8eb8-eba9222ef3ac admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:51.785074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee10df-4895-4b33-8eb4-af11c36004a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:51.785874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee10df-4895-4b33-8eb4-af11c36004a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:51.786034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee10df-4895-4b33-8eb4-af11c36004a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:51.786834 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee10df-4895-4b33-8eb4-af11c36004a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:51.787225 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee10df-4895-4b33-8eb4-af11c36004a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:51.788899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee10df-4895-4b33-8eb4-af11c36004a5 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:51.801262 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1385/2762] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:04:51.805598 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d31cc-b285-435d-a85d-02518ef585dd admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:51.817722 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d31cc-b285-435d-a85d-02518ef585dd admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:51.822926 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209fbde-bef3-4c05-84ef-8cd1474f8868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:51.823834 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209fbde-bef3-4c05-84ef-8cd1474f8868 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:51.824122 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209fbde-bef3-4c05-84ef-8cd1474f8868 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:51.824407 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209fbde-bef3-4c05-84ef-8cd1474f8868 admin admin] PATH_INFO: `/v3/projects/1706fc562cb44a5cad5cfab887fe1179/users/ce6a062bc8fc4915b70256b47bd9d5c1/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:51.826548 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209fbde-bef3-4c05-84ef-8cd1474f8868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1706fc562cb44a5cad5cfab887fe1179, user_id=ce6a062bc8fc4915b70256b47bd9d5c1, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:51.829311 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209fbde-bef3-4c05-84ef-8cd1474f8868 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:51.841918 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1378/2763] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:04:51 2022] PUT /identity/v3/projects/1706fc562cb44a5cad5cfab887fe1179/users/ce6a062bc8fc4915b70256b47bd9d5c1/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:51.853955 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df66ff1-e5bd-4449-9b9e-5143ab5ed3a1 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:51.854297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df66ff1-e5bd-4449-9b9e-5143ab5ed3a1 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:51.854719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df66ff1-e5bd-4449-9b9e-5143ab5ed3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:51.865432 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:51.894054 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-6df66ff1-e5bd-4449-9b9e-5143ab5ed3a1 None None] MFA Rules not processed for user `ce6a062bc8fc4915b70256b47bd9d5c1`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:51.896106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-6df66ff1-e5bd-4449-9b9e-5143ab5ed3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:52.014976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1386/2764] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:51 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:04:52.028020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209fbde-bef3-4c05-84ef-8cd1474f8868 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:52.030117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-0209fbde-bef3-4c05-84ef-8cd1474f8868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:52.036513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:52.105551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209fbde-bef3-4c05-84ef-8cd1474f8868 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:52.109366 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056a2c7-31da-4798-b7f0-70ad4beecbac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:52.110018 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056a2c7-31da-4798-b7f0-70ad4beecbac service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:52.110309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056a2c7-31da-4798-b7f0-70ad4beecbac service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:52.110581 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056a2c7-31da-4798-b7f0-70ad4beecbac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:52.121886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056a2c7-31da-4798-b7f0-70ad4beecbac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:52.123617 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056a2c7-31da-4798-b7f0-70ad4beecbac service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:52.135703 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1379/2765] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:04:52 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:04:52.239190 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df66ff1-e5bd-4449-9b9e-5143ab5ed3a1 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:52.249358 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df66ff1-e5bd-4449-9b9e-5143ab5ed3a1 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:52.258764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6eb9cd-a9ac-45e7-9c9b-8193f5de5ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:52.260017 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6eb9cd-a9ac-45e7-9c9b-8193f5de5ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:52.260550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6eb9cd-a9ac-45e7-9c9b-8193f5de5ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:52.261069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6eb9cd-a9ac-45e7-9c9b-8193f5de5ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:52.351047 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6eb9cd-a9ac-45e7-9c9b-8193f5de5ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:52.353543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6eb9cd-a9ac-45e7-9c9b-8193f5de5ac5 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:52.370187 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1387/2766] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:04:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:04:55.265037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056a2c7-31da-4798-b7f0-70ad4beecbac service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:55.337919 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056a2c7-31da-4798-b7f0-70ad4beecbac service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:55.342663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:55.343576 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:55.343850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:55.344121 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] PATH_INFO: `/v3/users/44d9dd5055224c5b8e11dba387dd98ed` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:55.345356 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44d9dd5055224c5b8e11dba387dd98ed)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:55.347973 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:55.409758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44d9dd5055224c5b8e11dba387dd98ed'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:55.418903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44d9dd5055224c5b8e11dba387dd98ed'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:55.419972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44d9dd5055224c5b8e11dba387dd98ed'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:55.431246 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44d9dd5055224c5b8e11dba387dd98ed'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:55.447545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44d9dd5055224c5b8e11dba387dd98ed'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:55.454322 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1380/2767] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:04:55 2022] DELETE /identity/v3/users/44d9dd5055224c5b8e11dba387dd98ed => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:55.460132 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f08764-76aa-428a-84fd-a951ad6574b2 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:55.460726 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f08764-76aa-428a-84fd-a951ad6574b2 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:55.461084 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f08764-76aa-428a-84fd-a951ad6574b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:55.493493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-c0f08764-76aa-428a-84fd-a951ad6574b2 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:55.495642 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c0f08764-76aa-428a-84fd-a951ad6574b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:55.580873 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1388/2768] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:04:55.591322 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:55.592992 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:55.641726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:55.683861 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc575907-bbf8-4cdf-9d9d-72a6f16b8fe6 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:55.688536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbef91c-90ad-4b21-bfff-3cdec3e7fa4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:55.689210 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbef91c-90ad-4b21-bfff-3cdec3e7fa4c service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:55.689499 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbef91c-90ad-4b21-bfff-3cdec3e7fa4c service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:55.689781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbef91c-90ad-4b21-bfff-3cdec3e7fa4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:55.704534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbef91c-90ad-4b21-bfff-3cdec3e7fa4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:55.706567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbef91c-90ad-4b21-bfff-3cdec3e7fa4c service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:55.717770 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1381/2769] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:04:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:04:56.081398 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f08764-76aa-428a-84fd-a951ad6574b2 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:56.083366 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c0f08764-76aa-428a-84fd-a951ad6574b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:56.105376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f08764-76aa-428a-84fd-a951ad6574b2 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:56.114444 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:56.115946 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:56.116417 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:56.116904 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] PATH_INFO: `/v3/projects/d8aa875187e545d996936eed6ff67c1c` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:56.118711 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8aa875187e545d996936eed6ff67c1c)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:56.122075 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:56.164780 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:56.201215 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8aa875187e545d996936eed6ff67c1c', 'request_id': 'req-f9abf1ea-cd63-44f9-8936-c45eda864d34'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:56.208297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8aa875187e545d996936eed6ff67c1c', 'request_id': 'req-f9abf1ea-cd63-44f9-8936-c45eda864d34'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:56.217160 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8aa875187e545d996936eed6ff67c1c', 'request_id': 'req-f9abf1ea-cd63-44f9-8936-c45eda864d34'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:56.230976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] The token cache is being invalidate because project d8aa875187e545d996936eed6ff67c1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:04:56.231313 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:56.231996 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:56.233593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1389/2770] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:04:56 2022] DELETE /identity/v3/projects/d8aa875187e545d996936eed6ff67c1c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:56.238131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbef91c-90ad-4b21-bfff-3cdec3e7fa4c service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:56.249222 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbef91c-90ad-4b21-bfff-3cdec3e7fa4c service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:56.321030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed9300-4a3a-4057-ac41-88b46340ff1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:56.322131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed9300-4a3a-4057-ac41-88b46340ff1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:56.322360 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed9300-4a3a-4057-ac41-88b46340ff1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:56.322532 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed9300-4a3a-4057-ac41-88b46340ff1e admin admin] PATH_INFO: `/v3/users/ce6a062bc8fc4915b70256b47bd9d5c1` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:56.323816 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed9300-4a3a-4057-ac41-88b46340ff1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce6a062bc8fc4915b70256b47bd9d5c1)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:56.325767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed9300-4a3a-4057-ac41-88b46340ff1e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:56.396919 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-02ed9300-4a3a-4057-ac41-88b46340ff1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce6a062bc8fc4915b70256b47bd9d5c1'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:56.409845 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-02ed9300-4a3a-4057-ac41-88b46340ff1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce6a062bc8fc4915b70256b47bd9d5c1'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:56.410928 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-02ed9300-4a3a-4057-ac41-88b46340ff1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce6a062bc8fc4915b70256b47bd9d5c1'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:56.422272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-02ed9300-4a3a-4057-ac41-88b46340ff1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce6a062bc8fc4915b70256b47bd9d5c1'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:56.443163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-02ed9300-4a3a-4057-ac41-88b46340ff1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce6a062bc8fc4915b70256b47bd9d5c1'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:56.452961 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1382/2771] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:04:56 2022] DELETE /identity/v3/users/ce6a062bc8fc4915b70256b47bd9d5c1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:04:56.847426 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:56.849564 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:56.868293 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abf1ea-cd63-44f9-8936-c45eda864d34 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:56.958328 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:56.959881 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:56.960385 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:56.960923 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998 admin admin] PATH_INFO: `/v3/projects/1706fc562cb44a5cad5cfab887fe1179` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:56.963145 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1706fc562cb44a5cad5cfab887fe1179)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:56.966883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:57.051415 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1706fc562cb44a5cad5cfab887fe1179', 'request_id': 'req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:57.060565 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1706fc562cb44a5cad5cfab887fe1179', 'request_id': 'req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:57.074323 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1706fc562cb44a5cad5cfab887fe1179', 'request_id': 'req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:57.091966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998 admin admin] The token cache is being invalidate because project 1706fc562cb44a5cad5cfab887fe1179 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:04:57.092458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:57.093493 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9e5e446a-376f-4cc4-a6ae-a4a6b6564998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:57.095883 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1390/2772] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:04:56 2022] DELETE /identity/v3/projects/1706fc562cb44a5cad5cfab887fe1179 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:57.102952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e6773-c7fc-42cc-bdde-9254183a10b9 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:57.103247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e6773-c7fc-42cc-bdde-9254183a10b9 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:57.103582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e6773-c7fc-42cc-bdde-9254183a10b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:57.126744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-859e6773-c7fc-42cc-bdde-9254183a10b9 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:57.128625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-859e6773-c7fc-42cc-bdde-9254183a10b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:57.181465 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:57.219245 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1383/2773] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:04:57.230388 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd1717-7899-41d7-b462-8e14b185994d None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:57.230741 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd1717-7899-41d7-b462-8e14b185994d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:57.231196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd1717-7899-41d7-b462-8e14b185994d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:57.251950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-35cd1717-7899-41d7-b462-8e14b185994d None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:57.253243 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-35cd1717-7899-41d7-b462-8e14b185994d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:57.263828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1391/2774] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:04:57.268577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e6773-c7fc-42cc-bdde-9254183a10b9 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:57.279167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e6773-c7fc-42cc-bdde-9254183a10b9 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:57.286043 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3033287-cadb-4b12-8a6a-24288c5df26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:57.287344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3033287-cadb-4b12-8a6a-24288c5df26a admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:57.287784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3033287-cadb-4b12-8a6a-24288c5df26a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:57.288232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3033287-cadb-4b12-8a6a-24288c5df26a admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:57.289336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3033287-cadb-4b12-8a6a-24288c5df26a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:04:57.289802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3033287-cadb-4b12-8a6a-24288c5df26a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:57.291685 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3033287-cadb-4b12-8a6a-24288c5df26a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:57.307094 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1384/2775] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:04:57.311612 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd1717-7899-41d7-b462-8e14b185994d None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:57.322782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd1717-7899-41d7-b462-8e14b185994d None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:57.327739 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3850dbc-c10c-498a-8920-097c812c12b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:57.328503 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3850dbc-c10c-498a-8920-097c812c12b3 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:57.328826 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3850dbc-c10c-498a-8920-097c812c12b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:57.329165 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3850dbc-c10c-498a-8920-097c812c12b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:57.329993 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3850dbc-c10c-498a-8920-097c812c12b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:57.332736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3850dbc-c10c-498a-8920-097c812c12b3 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:57.364471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1392/2776] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:04:57 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:04:57.368855 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3033287-cadb-4b12-8a6a-24288c5df26a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:57.381013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3033287-cadb-4b12-8a6a-24288c5df26a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:57.459583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fec11-cfe1-4629-83de-1f8bc8ebe6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:57.460699 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fec11-cfe1-4629-83de-1f8bc8ebe6bd admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:57.461188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fec11-cfe1-4629-83de-1f8bc8ebe6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:57.461651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fec11-cfe1-4629-83de-1f8bc8ebe6bd admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:57.462981 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fec11-cfe1-4629-83de-1f8bc8ebe6bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:57.466069 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fec11-cfe1-4629-83de-1f8bc8ebe6bd admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:57.490337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1385/2777] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:04:57 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:04:57.495015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3850dbc-c10c-498a-8920-097c812c12b3 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:57.506605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3850dbc-c10c-498a-8920-097c812c12b3 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:57.512231 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f73b4-a713-4675-b317-72bea8677e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:57.512974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f73b4-a713-4675-b317-72bea8677e5d admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:57.513328 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f73b4-a713-4675-b317-72bea8677e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:57.513691 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f73b4-a713-4675-b317-72bea8677e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:57.514503 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f73b4-a713-4675-b317-72bea8677e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:57.516246 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f73b4-a713-4675-b317-72bea8677e5d admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:57.534961 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1393/2778] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:04:57.539396 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fec11-cfe1-4629-83de-1f8bc8ebe6bd admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:57.551921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fec11-cfe1-4629-83de-1f8bc8ebe6bd admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:57.561558 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11116392-9002-44fa-99fd-08a649bd53dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:57.562257 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11116392-9002-44fa-99fd-08a649bd53dc admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:57.562542 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11116392-9002-44fa-99fd-08a649bd53dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:57.562821 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11116392-9002-44fa-99fd-08a649bd53dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:57.563605 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11116392-9002-44fa-99fd-08a649bd53dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:57.565488 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11116392-9002-44fa-99fd-08a649bd53dc admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:57.577223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1386/2779] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:04:57.582772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f73b4-a713-4675-b317-72bea8677e5d admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:57.589206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:57.596615 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f73b4-a713-4675-b317-72bea8677e5d admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:57.603629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac54e8-4345-40ea-bf8c-3ca7efa7b95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:57.604596 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac54e8-4345-40ea-bf8c-3ca7efa7b95c admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:57.604946 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac54e8-4345-40ea-bf8c-3ca7efa7b95c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:57.605296 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac54e8-4345-40ea-bf8c-3ca7efa7b95c admin admin] PATH_INFO: `/v3/projects/58f89612f31348f79e9004ed859715cc/users/25bbfe453a1543a2b77087d0d2723d17/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:57.623522 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac54e8-4345-40ea-bf8c-3ca7efa7b95c admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f89612f31348f79e9004ed859715cc, user_id=25bbfe453a1543a2b77087d0d2723d17, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:57.626219 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac54e8-4345-40ea-bf8c-3ca7efa7b95c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:57.649377 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1394/2780] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:04:57 2022] PUT /identity/v3/projects/58f89612f31348f79e9004ed859715cc/users/25bbfe453a1543a2b77087d0d2723d17/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:04:57.663042 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61a552-d7e3-47d4-a055-cead52a219a1 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:57.663651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61a552-d7e3-47d4-a055-cead52a219a1 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:57.664093 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61a552-d7e3-47d4-a055-cead52a219a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:57.699526 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3d61a552-d7e3-47d4-a055-cead52a219a1 None None] MFA Rules not processed for user `25bbfe453a1543a2b77087d0d2723d17`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:57.700900 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3d61a552-d7e3-47d4-a055-cead52a219a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:57.789698 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1387/2781] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:04:57.795139 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac54e8-4345-40ea-bf8c-3ca7efa7b95c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:57.877954 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac54e8-4345-40ea-bf8c-3ca7efa7b95c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:57.884147 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e3ffc-24b9-43f1-a327-acb25b809320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:57.884957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e3ffc-24b9-43f1-a327-acb25b809320 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:57.885356 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e3ffc-24b9-43f1-a327-acb25b809320 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:57.885729 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e3ffc-24b9-43f1-a327-acb25b809320 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:57.886537 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e3ffc-24b9-43f1-a327-acb25b809320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:57.888624 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e3ffc-24b9-43f1-a327-acb25b809320 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:57.917756 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1395/2782] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:04:57 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:04:57.923147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61a552-d7e3-47d4-a055-cead52a219a1 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:57.935153 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61a552-d7e3-47d4-a055-cead52a219a1 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:57.988636 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:58.017667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57711cab-7ac6-4e04-b875-e512dae92275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:58.019420 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57711cab-7ac6-4e04-b875-e512dae92275 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:58.019718 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57711cab-7ac6-4e04-b875-e512dae92275 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:58.020182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57711cab-7ac6-4e04-b875-e512dae92275 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:58.021385 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57711cab-7ac6-4e04-b875-e512dae92275 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:58.024820 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57711cab-7ac6-4e04-b875-e512dae92275 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:58.058160 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1388/2783] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:04:57 2022] POST /identity/v3/users => generated 299 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:04:58.062756 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e3ffc-24b9-43f1-a327-acb25b809320 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:58.076178 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e3ffc-24b9-43f1-a327-acb25b809320 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:58.082393 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1bbcc-ef41-43c6-b4ba-378ebc0ae134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:58.083128 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1bbcc-ef41-43c6-b4ba-378ebc0ae134 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:58.083420 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1bbcc-ef41-43c6-b4ba-378ebc0ae134 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:58.083705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1bbcc-ef41-43c6-b4ba-378ebc0ae134 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:58.084443 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1bbcc-ef41-43c6-b4ba-378ebc0ae134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:58.086082 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1bbcc-ef41-43c6-b4ba-378ebc0ae134 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:58.099012 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1396/2784] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:04:58.104213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57711cab-7ac6-4e04-b875-e512dae92275 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:58.119801 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57711cab-7ac6-4e04-b875-e512dae92275 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:58.126251 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af73558-a843-4db1-b44f-3fc6488407f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:58.127197 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af73558-a843-4db1-b44f-3fc6488407f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:58.127507 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af73558-a843-4db1-b44f-3fc6488407f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:58.127790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af73558-a843-4db1-b44f-3fc6488407f9 admin admin] PATH_INFO: `/v3/projects/b5ea4455b9224f7e97b7bdfdd0fb79fb/users/3796f7987a4845718c4a3442549826de/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:58.146829 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af73558-a843-4db1-b44f-3fc6488407f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ea4455b9224f7e97b7bdfdd0fb79fb, user_id=3796f7987a4845718c4a3442549826de, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:58.149651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af73558-a843-4db1-b44f-3fc6488407f9 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:58.167972 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1389/2785] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:04:58 2022] PUT /identity/v3/projects/b5ea4455b9224f7e97b7bdfdd0fb79fb/users/3796f7987a4845718c4a3442549826de/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:58.172678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1bbcc-ef41-43c6-b4ba-378ebc0ae134 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:58.184322 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1bbcc-ef41-43c6-b4ba-378ebc0ae134 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:58.261579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d005d-b6ba-4de2-99ba-1a9e50797cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:58.262641 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d005d-b6ba-4de2-99ba-1a9e50797cfd admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:58.263239 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d005d-b6ba-4de2-99ba-1a9e50797cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:58.263736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d005d-b6ba-4de2-99ba-1a9e50797cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:58.264966 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d005d-b6ba-4de2-99ba-1a9e50797cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:58.267542 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d005d-b6ba-4de2-99ba-1a9e50797cfd admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:58.284944 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1397/2786] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:04:58.291276 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af73558-a843-4db1-b44f-3fc6488407f9 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:58.306367 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af73558-a843-4db1-b44f-3fc6488407f9 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:58.313983 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d97a6-1728-42bd-a327-372210e258ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:58.316159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d97a6-1728-42bd-a327-372210e258ea admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:58.316643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d97a6-1728-42bd-a327-372210e258ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:58.317114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d97a6-1728-42bd-a327-372210e258ea admin admin] PATH_INFO: `/v3/projects/b5ea4455b9224f7e97b7bdfdd0fb79fb/users/3796f7987a4845718c4a3442549826de/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:58.320038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d97a6-1728-42bd-a327-372210e258ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ea4455b9224f7e97b7bdfdd0fb79fb, user_id=3796f7987a4845718c4a3442549826de, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:58.323329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d97a6-1728-42bd-a327-372210e258ea admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:58.335809 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1390/2787] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:04:58 2022] PUT /identity/v3/projects/b5ea4455b9224f7e97b7bdfdd0fb79fb/users/3796f7987a4845718c4a3442549826de/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:04:58.348416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620d95b-9216-4328-867b-16d3520567a5 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:58.349141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620d95b-9216-4328-867b-16d3520567a5 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:58.349777 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620d95b-9216-4328-867b-16d3520567a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:58.393668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-8620d95b-9216-4328-867b-16d3520567a5 None None] MFA Rules not processed for user `3796f7987a4845718c4a3442549826de`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:58.395837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8620d95b-9216-4328-867b-16d3520567a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:58.463424 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:58.522784 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1398/2788] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:04:58.543167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d97a6-1728-42bd-a327-372210e258ea admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:58.545721 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-d73d97a6-1728-42bd-a327-372210e258ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:58.636223 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d97a6-1728-42bd-a327-372210e258ea admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:58.641754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd37a6-be46-4e9f-bc96-84fa5fcda6b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:58.642805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd37a6-be46-4e9f-bc96-84fa5fcda6b8 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:58.643843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd37a6-be46-4e9f-bc96-84fa5fcda6b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:58.644364 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd37a6-be46-4e9f-bc96-84fa5fcda6b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:58.656483 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd37a6-be46-4e9f-bc96-84fa5fcda6b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:58.657958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd37a6-be46-4e9f-bc96-84fa5fcda6b8 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:58.670114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1391/2789] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:04:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:04:58.700860 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620d95b-9216-4328-867b-16d3520567a5 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:58.818069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620d95b-9216-4328-867b-16d3520567a5 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:58.824720 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:58.825633 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:58.825914 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:58.826194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] PATH_INFO: `/v3/users/25bbfe453a1543a2b77087d0d2723d17` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:58.827594 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25bbfe453a1543a2b77087d0d2723d17)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:58.829543 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:58.887051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25bbfe453a1543a2b77087d0d2723d17'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:58.896972 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25bbfe453a1543a2b77087d0d2723d17'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:58.906311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25bbfe453a1543a2b77087d0d2723d17'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:58.907471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25bbfe453a1543a2b77087d0d2723d17'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:58.908134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25bbfe453a1543a2b77087d0d2723d17'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:58.930841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1399/2790] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:04:58 2022] DELETE /identity/v3/users/25bbfe453a1543a2b77087d0d2723d17 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:58.937670 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def55cc-9cc2-4ccd-b899-b88385c34bfa None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:58.938131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def55cc-9cc2-4ccd-b899-b88385c34bfa None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:58.938582 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def55cc-9cc2-4ccd-b899-b88385c34bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:58.967902 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-2def55cc-9cc2-4ccd-b899-b88385c34bfa None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:04:58.969470 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2def55cc-9cc2-4ccd-b899-b88385c34bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:59.067225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1392/2791] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:04:59.079155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:59.081138 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:59.181448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bf832-c3d6-467c-831a-829e81348086 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:59.185421 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e7f5f-feb0-4d84-a08e-51bdfb783e4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:59.186084 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e7f5f-feb0-4d84-a08e-51bdfb783e4a service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:59.186374 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e7f5f-feb0-4d84-a08e-51bdfb783e4a service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:59.186654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e7f5f-feb0-4d84-a08e-51bdfb783e4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:59.195747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:59.200297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e7f5f-feb0-4d84-a08e-51bdfb783e4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:59.203250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e7f5f-feb0-4d84-a08e-51bdfb783e4a service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:59.216629 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1400/2792] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:04:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:04:59.596751 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def55cc-9cc2-4ccd-b899-b88385c34bfa None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:59.598853 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2def55cc-9cc2-4ccd-b899-b88385c34bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:04:59.619056 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def55cc-9cc2-4ccd-b899-b88385c34bfa None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:59.624909 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:59.625797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:59.626067 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:59.626384 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] PATH_INFO: `/v3/projects/58f89612f31348f79e9004ed859715cc` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:59.627919 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58f89612f31348f79e9004ed859715cc)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:59.629873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:59.682601 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:04:59.708807 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58f89612f31348f79e9004ed859715cc', 'request_id': 'req-14c097c1-f189-4ba1-9ac1-82b88be94016'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:59.719063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58f89612f31348f79e9004ed859715cc', 'request_id': 'req-14c097c1-f189-4ba1-9ac1-82b88be94016'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:59.727889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58f89612f31348f79e9004ed859715cc', 'request_id': 'req-14c097c1-f189-4ba1-9ac1-82b88be94016'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:59.742728 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] The token cache is being invalidate because project 58f89612f31348f79e9004ed859715cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:04:59.743251 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:59.744299 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:59.746787 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1393/2793] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:04:59 2022] DELETE /identity/v3/projects/58f89612f31348f79e9004ed859715cc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:04:59.751714 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e7f5f-feb0-4d84-a08e-51bdfb783e4a service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:04:59.763453 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e7f5f-feb0-4d84-a08e-51bdfb783e4a service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:04:59.852964 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cf470-2da8-46b6-a997-d97f3356a249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:04:59.853732 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cf470-2da8-46b6-a997-d97f3356a249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:04:59.854015 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cf470-2da8-46b6-a997-d97f3356a249 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:04:59.854287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cf470-2da8-46b6-a997-d97f3356a249 admin admin] PATH_INFO: `/v3/users/3796f7987a4845718c4a3442549826de` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:04:59.855666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cf470-2da8-46b6-a997-d97f3356a249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3796f7987a4845718c4a3442549826de)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:04:59.857604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cf470-2da8-46b6-a997-d97f3356a249 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:04:59.932521 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ee5cf470-2da8-46b6-a997-d97f3356a249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3796f7987a4845718c4a3442549826de'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:59.943457 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ee5cf470-2da8-46b6-a997-d97f3356a249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3796f7987a4845718c4a3442549826de'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:59.951693 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ee5cf470-2da8-46b6-a997-d97f3356a249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3796f7987a4845718c4a3442549826de'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:59.952535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ee5cf470-2da8-46b6-a997-d97f3356a249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3796f7987a4845718c4a3442549826de'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:59.953186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ee5cf470-2da8-46b6-a997-d97f3356a249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3796f7987a4845718c4a3442549826de'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:04:59.973614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1401/2794] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:04:59 2022] DELETE /identity/v3/users/3796f7987a4845718c4a3442549826de => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:00.298163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:00.300055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:00.320323 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c097c1-f189-4ba1-9ac1-82b88be94016 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:00.401930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70c849-dbf2-4396-afe9-2d6b397470a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:00.403612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70c849-dbf2-4396-afe9-2d6b397470a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:00.403612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70c849-dbf2-4396-afe9-2d6b397470a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:00.403612 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70c849-dbf2-4396-afe9-2d6b397470a0 admin admin] PATH_INFO: `/v3/projects/b5ea4455b9224f7e97b7bdfdd0fb79fb` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:00.404640 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70c849-dbf2-4396-afe9-2d6b397470a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ea4455b9224f7e97b7bdfdd0fb79fb)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:00.406584 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70c849-dbf2-4396-afe9-2d6b397470a0 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:00.485500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0e70c849-dbf2-4396-afe9-2d6b397470a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ea4455b9224f7e97b7bdfdd0fb79fb', 'request_id': 'req-0e70c849-dbf2-4396-afe9-2d6b397470a0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:00.494603 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0e70c849-dbf2-4396-afe9-2d6b397470a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ea4455b9224f7e97b7bdfdd0fb79fb', 'request_id': 'req-0e70c849-dbf2-4396-afe9-2d6b397470a0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:00.506958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0e70c849-dbf2-4396-afe9-2d6b397470a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ea4455b9224f7e97b7bdfdd0fb79fb', 'request_id': 'req-0e70c849-dbf2-4396-afe9-2d6b397470a0'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:00.524901 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0e70c849-dbf2-4396-afe9-2d6b397470a0 admin admin] The token cache is being invalidate because project b5ea4455b9224f7e97b7bdfdd0fb79fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:05:00.525347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0e70c849-dbf2-4396-afe9-2d6b397470a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:00.526337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-0e70c849-dbf2-4396-afe9-2d6b397470a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:00.528839 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1394/2795] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:05:00 2022] DELETE /identity/v3/projects/b5ea4455b9224f7e97b7bdfdd0fb79fb => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:00.535222 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecd1a4-8671-4746-8b02-a0d6a04e3321 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:00.535684 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecd1a4-8671-4746-8b02-a0d6a04e3321 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:00.536093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecd1a4-8671-4746-8b02-a0d6a04e3321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:00.557731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-5eecd1a4-8671-4746-8b02-a0d6a04e3321 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:00.559101 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-5eecd1a4-8671-4746-8b02-a0d6a04e3321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:00.661793 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1402/2796] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:00.669871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374f502-c6f1-41bd-9655-699b8cc159fa None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:00.670465 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374f502-c6f1-41bd-9655-699b8cc159fa None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:00.670583 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374f502-c6f1-41bd-9655-699b8cc159fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:00.692189 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-c374f502-c6f1-41bd-9655-699b8cc159fa None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:00.693434 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c374f502-c6f1-41bd-9655-699b8cc159fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:00.702066 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1395/2797] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:00.707157 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecd1a4-8671-4746-8b02-a0d6a04e3321 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:00.717808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecd1a4-8671-4746-8b02-a0d6a04e3321 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:00.722851 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a4a8d-e92c-436c-ac59-ad1d5145ec01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:00.723550 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a4a8d-e92c-436c-ac59-ad1d5145ec01 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:00.723839 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a4a8d-e92c-436c-ac59-ad1d5145ec01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:00.724114 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a4a8d-e92c-436c-ac59-ad1d5145ec01 admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:00.724853 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a4a8d-e92c-436c-ac59-ad1d5145ec01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:05:00.725159 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a4a8d-e92c-436c-ac59-ad1d5145ec01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:00.726763 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a4a8d-e92c-436c-ac59-ad1d5145ec01 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:00.739812 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1403/2798] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:05:00.745942 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374f502-c6f1-41bd-9655-699b8cc159fa None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:00.759849 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374f502-c6f1-41bd-9655-699b8cc159fa None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:00.764936 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46ae09-f0ce-4c69-89cf-95bc2283a12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:00.765626 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46ae09-f0ce-4c69-89cf-95bc2283a12d admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:00.765914 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46ae09-f0ce-4c69-89cf-95bc2283a12d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:00.766199 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46ae09-f0ce-4c69-89cf-95bc2283a12d admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:00.766925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46ae09-f0ce-4c69-89cf-95bc2283a12d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:00.769406 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46ae09-f0ce-4c69-89cf-95bc2283a12d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:00.798468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1396/2799] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:05:00 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:05:00.805719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a4a8d-e92c-436c-ac59-ad1d5145ec01 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:00.813314 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:00.820242 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a4a8d-e92c-436c-ac59-ad1d5145ec01 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:00.889483 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b904a-8b05-4e21-8408-d126d46b42a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:00.890212 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b904a-8b05-4e21-8408-d126d46b42a1 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:00.890547 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b904a-8b05-4e21-8408-d126d46b42a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:00.890888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b904a-8b05-4e21-8408-d126d46b42a1 admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:00.891756 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b904a-8b05-4e21-8408-d126d46b42a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:00.893739 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b904a-8b05-4e21-8408-d126d46b42a1 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:00.922661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1404/2800] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:05:00 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:05:00.928651 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46ae09-f0ce-4c69-89cf-95bc2283a12d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:00.939944 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46ae09-f0ce-4c69-89cf-95bc2283a12d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:00.945122 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edcf43-7197-4e18-abf9-1923c46a404e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:00.945843 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edcf43-7197-4e18-abf9-1923c46a404e admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:00.946180 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edcf43-7197-4e18-abf9-1923c46a404e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:00.946531 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edcf43-7197-4e18-abf9-1923c46a404e admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:00.947427 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57edcf43-7197-4e18-abf9-1923c46a404e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:00.949144 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57edcf43-7197-4e18-abf9-1923c46a404e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:00.961166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1397/2801] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:05:00.965590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b904a-8b05-4e21-8408-d126d46b42a1 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:00.976730 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b904a-8b05-4e21-8408-d126d46b42a1 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:00.981833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab6c5c-3daa-41a4-b417-53e3449b8d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:00.982548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab6c5c-3daa-41a4-b417-53e3449b8d63 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:00.982891 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab6c5c-3daa-41a4-b417-53e3449b8d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:00.983196 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab6c5c-3daa-41a4-b417-53e3449b8d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:00.983969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab6c5c-3daa-41a4-b417-53e3449b8d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:00.985634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab6c5c-3daa-41a4-b417-53e3449b8d63 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:00.997616 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1405/2802] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:05:01.003782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edcf43-7197-4e18-abf9-1923c46a404e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:01.014645 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edcf43-7197-4e18-abf9-1923c46a404e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:01.021610 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc015ab-5333-477e-9ce7-5199f7283dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:01.023118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc015ab-5333-477e-9ce7-5199f7283dbd admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:01.023588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc015ab-5333-477e-9ce7-5199f7283dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:01.024048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc015ab-5333-477e-9ce7-5199f7283dbd admin admin] PATH_INFO: `/v3/projects/a2d457c13eca4e7d96cfbee77e9b1bb7/users/1861b3c6dfa44c4a82fc15254148d75e/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:01.036726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:01.050341 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc015ab-5333-477e-9ce7-5199f7283dbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d457c13eca4e7d96cfbee77e9b1bb7, user_id=1861b3c6dfa44c4a82fc15254148d75e, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:01.053118 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc015ab-5333-477e-9ce7-5199f7283dbd admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:01.068020 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1398/2803] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:05:01 2022] PUT /identity/v3/projects/a2d457c13eca4e7d96cfbee77e9b1bb7/users/1861b3c6dfa44c4a82fc15254148d75e/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:01.078523 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66f387-85b6-49eb-a51f-da4660273f92 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:01.078815 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66f387-85b6-49eb-a51f-da4660273f92 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:01.079141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66f387-85b6-49eb-a51f-da4660273f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:01.113690 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-dc66f387-85b6-49eb-a51f-da4660273f92 None None] MFA Rules not processed for user `1861b3c6dfa44c4a82fc15254148d75e`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:01.114990 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-dc66f387-85b6-49eb-a51f-da4660273f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:01.189820 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1406/2804] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:01.194660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc015ab-5333-477e-9ce7-5199f7283dbd admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:01.267107 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc015ab-5333-477e-9ce7-5199f7283dbd admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:01.273128 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775e457-2443-4060-a8e4-8ac8e626518e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:01.273810 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775e457-2443-4060-a8e4-8ac8e626518e admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:01.274101 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775e457-2443-4060-a8e4-8ac8e626518e admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:01.274383 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775e457-2443-4060-a8e4-8ac8e626518e admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:01.275129 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775e457-2443-4060-a8e4-8ac8e626518e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:01.277044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775e457-2443-4060-a8e4-8ac8e626518e admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:01.306647 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1399/2805] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:05:01 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:05:01.310782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66f387-85b6-49eb-a51f-da4660273f92 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:01.323976 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66f387-85b6-49eb-a51f-da4660273f92 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:01.397701 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec583d5-0903-4c90-87f6-05535456008c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:01.398390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec583d5-0903-4c90-87f6-05535456008c admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:01.398675 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec583d5-0903-4c90-87f6-05535456008c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:01.398988 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec583d5-0903-4c90-87f6-05535456008c admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:01.399886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec583d5-0903-4c90-87f6-05535456008c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:01.402517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec583d5-0903-4c90-87f6-05535456008c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:01.423341 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:01.437266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1407/2806] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:05:01 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:05:01.443052 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775e457-2443-4060-a8e4-8ac8e626518e admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:01.464818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775e457-2443-4060-a8e4-8ac8e626518e admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:01.470185 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edb4c5-868d-46af-8f93-d0e1ef8f7987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:01.471609 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edb4c5-868d-46af-8f93-d0e1ef8f7987 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:01.471958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edb4c5-868d-46af-8f93-d0e1ef8f7987 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:01.472320 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edb4c5-868d-46af-8f93-d0e1ef8f7987 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:01.473107 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edb4c5-868d-46af-8f93-d0e1ef8f7987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:01.474840 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edb4c5-868d-46af-8f93-d0e1ef8f7987 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:01.487547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1400/2807] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:05:01.492768 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec583d5-0903-4c90-87f6-05535456008c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:01.507164 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec583d5-0903-4c90-87f6-05535456008c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:01.512349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d855ec8-2386-4221-b15d-d4755b94973d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:01.513253 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d855ec8-2386-4221-b15d-d4755b94973d admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:01.513532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d855ec8-2386-4221-b15d-d4755b94973d admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:01.513810 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d855ec8-2386-4221-b15d-d4755b94973d admin admin] PATH_INFO: `/v3/projects/9d5fdf15e0b3463db6ae2eb41192e27c/users/755b48a47b95424496e75e137d209fe7/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:01.534705 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d855ec8-2386-4221-b15d-d4755b94973d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d5fdf15e0b3463db6ae2eb41192e27c, user_id=755b48a47b95424496e75e137d209fe7, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:01.540579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d855ec8-2386-4221-b15d-d4755b94973d admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:01.564390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1408/2808] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:05:01 2022] PUT /identity/v3/projects/9d5fdf15e0b3463db6ae2eb41192e27c/users/755b48a47b95424496e75e137d209fe7/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:01.568476 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edb4c5-868d-46af-8f93-d0e1ef8f7987 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:01.578904 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edb4c5-868d-46af-8f93-d0e1ef8f7987 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:01.645678 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8a60b-5f24-4fcb-aa6b-b09716735e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:01.646993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a60b-5f24-4fcb-aa6b-b09716735e71 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:01.647501 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a60b-5f24-4fcb-aa6b-b09716735e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:01.648044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a60b-5f24-4fcb-aa6b-b09716735e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:01.649085 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8a60b-5f24-4fcb-aa6b-b09716735e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:01.651744 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8a60b-5f24-4fcb-aa6b-b09716735e71 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:01.666534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1401/2809] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:05:01.671011 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d855ec8-2386-4221-b15d-d4755b94973d admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:01.683329 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d855ec8-2386-4221-b15d-d4755b94973d admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:01.688546 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fe4de-3f69-4098-8cbd-1f2152cd3c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:01.689466 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fe4de-3f69-4098-8cbd-1f2152cd3c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:01.689806 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fe4de-3f69-4098-8cbd-1f2152cd3c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:01.690090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fe4de-3f69-4098-8cbd-1f2152cd3c04 admin admin] PATH_INFO: `/v3/projects/9d5fdf15e0b3463db6ae2eb41192e27c/users/755b48a47b95424496e75e137d209fe7/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:01.692468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fe4de-3f69-4098-8cbd-1f2152cd3c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d5fdf15e0b3463db6ae2eb41192e27c, user_id=755b48a47b95424496e75e137d209fe7, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:01.695184 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fe4de-3f69-4098-8cbd-1f2152cd3c04 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:01.706360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1409/2810] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:05:01 2022] PUT /identity/v3/projects/9d5fdf15e0b3463db6ae2eb41192e27c/users/755b48a47b95424496e75e137d209fe7/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:05:01.785382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a076d2c-7012-4e97-9f9e-93001401d3d5 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:01.785922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a076d2c-7012-4e97-9f9e-93001401d3d5 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:01.785922 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a076d2c-7012-4e97-9f9e-93001401d3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:01.814452 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-5a076d2c-7012-4e97-9f9e-93001401d3d5 None None] MFA Rules not processed for user `755b48a47b95424496e75e137d209fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:01.815750 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5a076d2c-7012-4e97-9f9e-93001401d3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:01.862929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:01.890433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1402/2811] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:01.904258 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fe4de-3f69-4098-8cbd-1f2152cd3c04 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:01.905938 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-394fe4de-3f69-4098-8cbd-1f2152cd3c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:01.976357 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fe4de-3f69-4098-8cbd-1f2152cd3c04 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:01.980369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b78dcb-7924-4e17-a97c-b8f54749b8c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:01.981067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b78dcb-7924-4e17-a97c-b8f54749b8c7 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:01.981354 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b78dcb-7924-4e17-a97c-b8f54749b8c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:01.981635 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b78dcb-7924-4e17-a97c-b8f54749b8c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:01.991989 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b78dcb-7924-4e17-a97c-b8f54749b8c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:01.993442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b78dcb-7924-4e17-a97c-b8f54749b8c7 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:02.004119 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1410/2812] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:05:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:05:02.091002 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a076d2c-7012-4e97-9f9e-93001401d3d5 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:02.102577 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a076d2c-7012-4e97-9f9e-93001401d3d5 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:02.106702 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da269a94-8fff-4541-b096-2efcdbcabc20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:02.107388 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da269a94-8fff-4541-b096-2efcdbcabc20 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:02.107669 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da269a94-8fff-4541-b096-2efcdbcabc20 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:02.107946 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da269a94-8fff-4541-b096-2efcdbcabc20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:02.180103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da269a94-8fff-4541-b096-2efcdbcabc20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:02.181547 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da269a94-8fff-4541-b096-2efcdbcabc20 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:02.191811 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1403/2813] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:05:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:05:06.889746 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b78dcb-7924-4e17-a97c-b8f54749b8c7 service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:06.953399 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b78dcb-7924-4e17-a97c-b8f54749b8c7 service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:06.958520 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:06.960656 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:06.960939 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:06.961214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] PATH_INFO: `/v3/users/1861b3c6dfa44c4a82fc15254148d75e` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:06.963634 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1861b3c6dfa44c4a82fc15254148d75e)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:06.965535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:07.012570 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1861b3c6dfa44c4a82fc15254148d75e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.020229 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1861b3c6dfa44c4a82fc15254148d75e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.026990 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1861b3c6dfa44c4a82fc15254148d75e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.027618 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1861b3c6dfa44c4a82fc15254148d75e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.028214 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1861b3c6dfa44c4a82fc15254148d75e'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.032648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:07.048617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1411/2814] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:05:06 2022] DELETE /identity/v3/users/1861b3c6dfa44c4a82fc15254148d75e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:07.053759 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b1cf8-3264-4a32-b549-c16526b66f43 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:07.054278 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b1cf8-3264-4a32-b549-c16526b66f43 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:07.054392 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b1cf8-3264-4a32-b549-c16526b66f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:07.073725 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-ef4b1cf8-3264-4a32-b549-c16526b66f43 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:07.075337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ef4b1cf8-3264-4a32-b549-c16526b66f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:07.138371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1404/2815] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:07.148300 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:07.149719 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:07.208722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae473fc-8e84-4cc4-b0b3-d5421f41eafd admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:07.212461 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7628a-2f0e-4647-8a0e-a7f978278e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:07.213105 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7628a-2f0e-4647-8a0e-a7f978278e0a service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:07.213390 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7628a-2f0e-4647-8a0e-a7f978278e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:07.213786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7628a-2f0e-4647-8a0e-a7f978278e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:07.223544 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7628a-2f0e-4647-8a0e-a7f978278e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:07.224963 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7628a-2f0e-4647-8a0e-a7f978278e0a service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:07.234316 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1412/2816] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:05:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:05:07.510635 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b1cf8-3264-4a32-b549-c16526b66f43 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:07.512099 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ef4b1cf8-3264-4a32-b549-c16526b66f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:07.525619 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b1cf8-3264-4a32-b549-c16526b66f43 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:07.530308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:07.531204 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:07.531489 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:07.531767 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] PATH_INFO: `/v3/projects/a2d457c13eca4e7d96cfbee77e9b1bb7` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:07.533013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d457c13eca4e7d96cfbee77e9b1bb7)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:07.535364 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:07.587191 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d457c13eca4e7d96cfbee77e9b1bb7', 'request_id': 'req-6bbb0c53-ca95-4de5-938f-580205163ce4'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.593422 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d457c13eca4e7d96cfbee77e9b1bb7', 'request_id': 'req-6bbb0c53-ca95-4de5-938f-580205163ce4'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.602815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d457c13eca4e7d96cfbee77e9b1bb7', 'request_id': 'req-6bbb0c53-ca95-4de5-938f-580205163ce4'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.613910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] The token cache is being invalidate because project a2d457c13eca4e7d96cfbee77e9b1bb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:05:07.614233 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.614903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.616468 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1405/2817] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:05:07 2022] DELETE /identity/v3/projects/a2d457c13eca4e7d96cfbee77e9b1bb7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:07.620578 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7628a-2f0e-4647-8a0e-a7f978278e0a service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:07.631133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7628a-2f0e-4647-8a0e-a7f978278e0a service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:07.698507 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71c080-aa85-425a-bef2-2d016de52a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:07.699406 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71c080-aa85-425a-bef2-2d016de52a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:07.699693 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71c080-aa85-425a-bef2-2d016de52a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:07.699969 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71c080-aa85-425a-bef2-2d016de52a69 admin admin] PATH_INFO: `/v3/users/755b48a47b95424496e75e137d209fe7` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:07.701164 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71c080-aa85-425a-bef2-2d016de52a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=755b48a47b95424496e75e137d209fe7)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:07.703194 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71c080-aa85-425a-bef2-2d016de52a69 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:07.749009 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-fd71c080-aa85-425a-bef2-2d016de52a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755b48a47b95424496e75e137d209fe7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.757351 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-fd71c080-aa85-425a-bef2-2d016de52a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755b48a47b95424496e75e137d209fe7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.765130 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-fd71c080-aa85-425a-bef2-2d016de52a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755b48a47b95424496e75e137d209fe7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.765795 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-fd71c080-aa85-425a-bef2-2d016de52a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755b48a47b95424496e75e137d209fe7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.766411 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-fd71c080-aa85-425a-bef2-2d016de52a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755b48a47b95424496e75e137d209fe7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:07.787121 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1413/2818] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:05:07 2022] DELETE /identity/v3/users/755b48a47b95424496e75e137d209fe7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:08.059790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:08.061251 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:08.073032 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:08.077559 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb0c53-ca95-4de5-938f-580205163ce4 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:08.137358 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:08.138221 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:08.138511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:08.138784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139 admin admin] PATH_INFO: `/v3/projects/9d5fdf15e0b3463db6ae2eb41192e27c` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:08.140119 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d5fdf15e0b3463db6ae2eb41192e27c)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:08.142009 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:08.188585 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d5fdf15e0b3463db6ae2eb41192e27c', 'request_id': 'req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:08.195279 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d5fdf15e0b3463db6ae2eb41192e27c', 'request_id': 'req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:08.202327 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d5fdf15e0b3463db6ae2eb41192e27c', 'request_id': 'req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:08.213534 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139 admin admin] The token cache is being invalidate because project 9d5fdf15e0b3463db6ae2eb41192e27c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:05:08.213818 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:08.214435 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-d4bda81a-d07a-4caf-bbfa-4d28fe85b139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:08.215999 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1406/2819] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:05:08 2022] DELETE /identity/v3/projects/9d5fdf15e0b3463db6ae2eb41192e27c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:08.221281 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302f7c8-d9f7-4a3d-b3c9-c1830c939298 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:08.221814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302f7c8-d9f7-4a3d-b3c9-c1830c939298 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:08.222055 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302f7c8-d9f7-4a3d-b3c9-c1830c939298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:08.234312 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:08.241217 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-7302f7c8-d9f7-4a3d-b3c9-c1830c939298 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:08.243092 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-7302f7c8-d9f7-4a3d-b3c9-c1830c939298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:08.305654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1414/2820] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:08.313161 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d93199-bd20-44da-9e9b-26f25ebcf800 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:08.313453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d93199-bd20-44da-9e9b-26f25ebcf800 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:08.313733 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d93199-bd20-44da-9e9b-26f25ebcf800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:08.333095 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-c2d93199-bd20-44da-9e9b-26f25ebcf800 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:08.334309 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-c2d93199-bd20-44da-9e9b-26f25ebcf800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:08.342302 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1407/2821] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:08.346935 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302f7c8-d9f7-4a3d-b3c9-c1830c939298 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:08.356841 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302f7c8-d9f7-4a3d-b3c9-c1830c939298 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:08.361686 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddbc0f-950e-4b51-a434-9f27bf7c37de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:08.362283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddbc0f-950e-4b51-a434-9f27bf7c37de admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:08.362566 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddbc0f-950e-4b51-a434-9f27bf7c37de admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:08.362839 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddbc0f-950e-4b51-a434-9f27bf7c37de admin admin] PATH_INFO: `/v3/domains` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:08.363592 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddbc0f-950e-4b51-a434-9f27bf7c37de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:05:08.363895 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddbc0f-950e-4b51-a434-9f27bf7c37de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:08.365471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddbc0f-950e-4b51-a434-9f27bf7c37de admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:08.376878 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1415/2822] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:05:08.381927 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d93199-bd20-44da-9e9b-26f25ebcf800 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:08.394497 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d93199-bd20-44da-9e9b-26f25ebcf800 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:08.399451 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829866f5-72a6-4d73-a908-5ccc6fa26263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:08.400228 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829866f5-72a6-4d73-a908-5ccc6fa26263 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:08.400397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829866f5-72a6-4d73-a908-5ccc6fa26263 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:08.400676 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829866f5-72a6-4d73-a908-5ccc6fa26263 admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:08.401382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829866f5-72a6-4d73-a908-5ccc6fa26263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:08.403317 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829866f5-72a6-4d73-a908-5ccc6fa26263 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:08.429147 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1408/2823] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:05:08 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:05:08.433340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddbc0f-950e-4b51-a434-9f27bf7c37de admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:08.444517 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddbc0f-950e-4b51-a434-9f27bf7c37de admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:08.499676 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085f93d-d075-4eb6-9a78-027850ad18ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:08.500416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085f93d-d075-4eb6-9a78-027850ad18ec admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:08.500697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085f93d-d075-4eb6-9a78-027850ad18ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:08.500974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085f93d-d075-4eb6-9a78-027850ad18ec admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:08.501844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085f93d-d075-4eb6-9a78-027850ad18ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:08.504311 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085f93d-d075-4eb6-9a78-027850ad18ec admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:08.524957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1416/2824] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:05:08 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:05:08.529227 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829866f5-72a6-4d73-a908-5ccc6fa26263 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:08.539562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829866f5-72a6-4d73-a908-5ccc6fa26263 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:08.544131 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf4b1c-a80d-444b-a1bc-5348fb84fdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:08.544812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf4b1c-a80d-444b-a1bc-5348fb84fdd0 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:08.545098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf4b1c-a80d-444b-a1bc-5348fb84fdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:08.545371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf4b1c-a80d-444b-a1bc-5348fb84fdd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:08.546089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf4b1c-a80d-444b-a1bc-5348fb84fdd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:08.547714 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf4b1c-a80d-444b-a1bc-5348fb84fdd0 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:08.558774 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1409/2825] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:05:08.562631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085f93d-d075-4eb6-9a78-027850ad18ec admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:08.573371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085f93d-d075-4eb6-9a78-027850ad18ec admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:08.577984 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d540fc-36db-43f9-9028-7c875ec635a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:08.578631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d540fc-36db-43f9-9028-7c875ec635a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:08.578960 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d540fc-36db-43f9-9028-7c875ec635a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:08.579227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d540fc-36db-43f9-9028-7c875ec635a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:08.579959 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d540fc-36db-43f9-9028-7c875ec635a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:08.582731 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d540fc-36db-43f9-9028-7c875ec635a5 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:08.594112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1417/2826] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:05:08.598271 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf4b1c-a80d-444b-a1bc-5348fb84fdd0 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:08.608313 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf4b1c-a80d-444b-a1bc-5348fb84fdd0 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:08.613523 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8a98d-5678-447a-a88c-f186e00f654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:08.614397 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8a98d-5678-447a-a88c-f186e00f654a admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:08.614685 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8a98d-5678-447a-a88c-f186e00f654a admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:08.615008 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8a98d-5678-447a-a88c-f186e00f654a admin admin] PATH_INFO: `/v3/projects/eada19daac084621ab61d9d920cf771f/users/f587432c85554cdb998da638eeb88581/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:08.630133 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8a98d-5678-447a-a88c-f186e00f654a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eada19daac084621ab61d9d920cf771f, user_id=f587432c85554cdb998da638eeb88581, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:08.633072 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8a98d-5678-447a-a88c-f186e00f654a admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:08.649382 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1410/2827] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:05:08 2022] PUT /identity/v3/projects/eada19daac084621ab61d9d920cf771f/users/f587432c85554cdb998da638eeb88581/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:08.660154 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821025d9-7b5c-4232-91f5-929c8695792c None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:08.660448 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821025d9-7b5c-4232-91f5-929c8695792c None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:08.660733 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821025d9-7b5c-4232-91f5-929c8695792c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:08.687953 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-821025d9-7b5c-4232-91f5-929c8695792c None None] MFA Rules not processed for user `f587432c85554cdb998da638eeb88581`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:08.689167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-821025d9-7b5c-4232-91f5-929c8695792c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:08.754349 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1418/2828] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:08.759121 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8a98d-5678-447a-a88c-f186e00f654a admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:08.793329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:08.826418 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8a98d-5678-447a-a88c-f186e00f654a admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:08.831103 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8988f-f209-407d-ab83-f0781e59794c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:08.831760 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8988f-f209-407d-ab83-f0781e59794c admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:08.832044 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8988f-f209-407d-ab83-f0781e59794c admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:08.832329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8988f-f209-407d-ab83-f0781e59794c admin admin] PATH_INFO: `/v3/projects` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:08.833047 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8988f-f209-407d-ab83-f0781e59794c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:08.834960 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8988f-f209-407d-ab83-f0781e59794c admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:08.858993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1411/2829] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:05:08 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:05:08.863643 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821025d9-7b5c-4232-91f5-929c8695792c None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:08.869043 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:08.876206 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821025d9-7b5c-4232-91f5-929c8695792c None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:08.932287 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f1056-a6a1-4852-b782-62a3ec0a972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:08.932952 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f1056-a6a1-4852-b782-62a3ec0a972c admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:08.933235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f1056-a6a1-4852-b782-62a3ec0a972c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:08.933512 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f1056-a6a1-4852-b782-62a3ec0a972c admin admin] PATH_INFO: `/v3/users` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:08.934203 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f1056-a6a1-4852-b782-62a3ec0a972c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:08.936133 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f1056-a6a1-4852-b782-62a3ec0a972c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:08.957014 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1419/2830] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:05:08 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:05:08.960985 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8988f-f209-407d-ab83-f0781e59794c admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:08.971545 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8988f-f209-407d-ab83-f0781e59794c admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:08.976374 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3df1aa-7c1f-44c7-98c5-1bcb9f120159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:08.977053 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3df1aa-7c1f-44c7-98c5-1bcb9f120159 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:08.977337 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3df1aa-7c1f-44c7-98c5-1bcb9f120159 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:08.977643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3df1aa-7c1f-44c7-98c5-1bcb9f120159 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:08.978377 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3df1aa-7c1f-44c7-98c5-1bcb9f120159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:08.980014 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3df1aa-7c1f-44c7-98c5-1bcb9f120159 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:08.991307 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1412/2831] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:05:08.995442 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f1056-a6a1-4852-b782-62a3ec0a972c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:09.006343 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f1056-a6a1-4852-b782-62a3ec0a972c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:09.011231 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f6771-0a3f-4ff8-9ec2-cb20c4621f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:09.012105 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f6771-0a3f-4ff8-9ec2-cb20c4621f95 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:09.012389 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f6771-0a3f-4ff8-9ec2-cb20c4621f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:09.012666 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f6771-0a3f-4ff8-9ec2-cb20c4621f95 admin admin] PATH_INFO: `/v3/projects/c290f53bf156420182f9c2111573d805/users/ec74b5e25b114474859180dc3de642f9/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:09.028097 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f6771-0a3f-4ff8-9ec2-cb20c4621f95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c290f53bf156420182f9c2111573d805, user_id=ec74b5e25b114474859180dc3de642f9, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:09.030739 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f6771-0a3f-4ff8-9ec2-cb20c4621f95 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:09.043933 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1420/2832] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:05:08 2022] PUT /identity/v3/projects/c290f53bf156420182f9c2111573d805/users/ec74b5e25b114474859180dc3de642f9/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:09.048312 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3df1aa-7c1f-44c7-98c5-1bcb9f120159 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:09.058302 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3df1aa-7c1f-44c7-98c5-1bcb9f120159 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:09.114642 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978db96-53a7-45e2-9e54-88a72f256e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:09.115334 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978db96-53a7-45e2-9e54-88a72f256e3b admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:09.115524 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978db96-53a7-45e2-9e54-88a72f256e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:09.115797 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978db96-53a7-45e2-9e54-88a72f256e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:09.116521 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978db96-53a7-45e2-9e54-88a72f256e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:09.118098 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978db96-53a7-45e2-9e54-88a72f256e3b admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:09.128873 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1413/2833] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:05:09.133942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f6771-0a3f-4ff8-9ec2-cb20c4621f95 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:09.145480 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f6771-0a3f-4ff8-9ec2-cb20c4621f95 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:09.150722 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cdfd1-d1f2-4c4a-8e7b-11b5b4f20965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:09.151667 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cdfd1-d1f2-4c4a-8e7b-11b5b4f20965 admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:09.151957 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cdfd1-d1f2-4c4a-8e7b-11b5b4f20965 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:09.152240 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cdfd1-d1f2-4c4a-8e7b-11b5b4f20965 admin admin] PATH_INFO: `/v3/projects/c290f53bf156420182f9c2111573d805/users/ec74b5e25b114474859180dc3de642f9/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:09.154234 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cdfd1-d1f2-4c4a-8e7b-11b5b4f20965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c290f53bf156420182f9c2111573d805, user_id=ec74b5e25b114474859180dc3de642f9, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:09.156897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cdfd1-d1f2-4c4a-8e7b-11b5b4f20965 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:09.167626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1421/2834] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:05:09 2022] PUT /identity/v3/projects/c290f53bf156420182f9c2111573d805/users/ec74b5e25b114474859180dc3de642f9/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:09.179268 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bf06e-90ad-4199-a8f5-1a7fab56decb None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:09.179564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bf06e-90ad-4199-a8f5-1a7fab56decb None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:09.179847 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bf06e-90ad-4199-a8f5-1a7fab56decb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:09.207812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6d8bf06e-90ad-4199-a8f5-1a7fab56decb None None] MFA Rules not processed for user `ec74b5e25b114474859180dc3de642f9`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:09.209049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6d8bf06e-90ad-4199-a8f5-1a7fab56decb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:09.276671 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1414/2835] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:09.287605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cdfd1-d1f2-4c4a-8e7b-11b5b4f20965 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:09.289339 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8d7cdfd1-d1f2-4c4a-8e7b-11b5b4f20965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:09.347631 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cdfd1-d1f2-4c4a-8e7b-11b5b4f20965 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:09.351247 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de69ec-a739-4c64-aa12-df9e52b324d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:09.351879 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de69ec-a739-4c64-aa12-df9e52b324d4 service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:09.352160 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de69ec-a739-4c64-aa12-df9e52b324d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:09.352437 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de69ec-a739-4c64-aa12-df9e52b324d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:09.361337 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de69ec-a739-4c64-aa12-df9e52b324d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:09.363431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de69ec-a739-4c64-aa12-df9e52b324d4 service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:09.372635 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1422/2836] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:05:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:05:09.728163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bf06e-90ad-4199-a8f5-1a7fab56decb None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:09.759331 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:09.793395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bf06e-90ad-4199-a8f5-1a7fab56decb None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:09.797958 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:09.798831 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:09.799272 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:09.799553 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] PATH_INFO: `/v3/users/f587432c85554cdb998da638eeb88581` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:09.800732 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f587432c85554cdb998da638eeb88581)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:09.802663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:09.848649 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f587432c85554cdb998da638eeb88581'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:09.855518 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f587432c85554cdb998da638eeb88581'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:09.856179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f587432c85554cdb998da638eeb88581'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:09.863575 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f587432c85554cdb998da638eeb88581'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:09.875191 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f587432c85554cdb998da638eeb88581'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:09.881564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1415/2837] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:05:09 2022] DELETE /identity/v3/users/f587432c85554cdb998da638eeb88581 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:09.886272 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b364601-0f79-47d4-974c-abadffe469c7 None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:09.886583 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b364601-0f79-47d4-974c-abadffe469c7 None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:09.886942 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b364601-0f79-47d4-974c-abadffe469c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:09.904609 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-3b364601-0f79-47d4-974c-abadffe469c7 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:09.905798 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3b364601-0f79-47d4-974c-abadffe469c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:09.956025 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:09.966068 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1423/2838] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:09.975731 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:09.977124 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:10.037159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6eed1f-6585-45cd-9c63-7be03fc5e28f admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:10.040885 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea2676-34b2-450f-9b70-f1657b7f7ec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:10.041528 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea2676-34b2-450f-9b70-f1657b7f7ec4 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:10.041815 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea2676-34b2-450f-9b70-f1657b7f7ec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:10.042092 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea2676-34b2-450f-9b70-f1657b7f7ec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:10.052424 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea2676-34b2-450f-9b70-f1657b7f7ec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:10.053850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea2676-34b2-450f-9b70-f1657b7f7ec4 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:10.063269 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1416/2839] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:05:10.323842 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b364601-0f79-47d4-974c-abadffe469c7 None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:10.325483 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3b364601-0f79-47d4-974c-abadffe469c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:10.339814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b364601-0f79-47d4-974c-abadffe469c7 None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:10.345200 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:10.346425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:10.346605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:10.346926 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] PATH_INFO: `/v3/projects/eada19daac084621ab61d9d920cf771f` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:10.348112 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eada19daac084621ab61d9d920cf771f)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:10.350147 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:10.399788 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eada19daac084621ab61d9d920cf771f', 'request_id': 'req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:10.405469 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eada19daac084621ab61d9d920cf771f', 'request_id': 'req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:10.413155 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eada19daac084621ab61d9d920cf771f', 'request_id': 'req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:10.425231 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] The token cache is being invalidate because project eada19daac084621ab61d9d920cf771f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:05:10.425519 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:10.426163 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:10.427747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1424/2840] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:05:10 2022] DELETE /identity/v3/projects/eada19daac084621ab61d9d920cf771f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:05:10.431778 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea2676-34b2-450f-9b70-f1657b7f7ec4 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:10.442051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea2676-34b2-450f-9b70-f1657b7f7ec4 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:10.486115 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:10.499725 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7deeff-07fe-4555-993a-ccf42bc79516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:10.500599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7deeff-07fe-4555-993a-ccf42bc79516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:10.500884 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7deeff-07fe-4555-993a-ccf42bc79516 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:10.501163 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7deeff-07fe-4555-993a-ccf42bc79516 admin admin] PATH_INFO: `/v3/users/ec74b5e25b114474859180dc3de642f9` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:10.502589 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7deeff-07fe-4555-993a-ccf42bc79516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec74b5e25b114474859180dc3de642f9)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:10.504530 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7deeff-07fe-4555-993a-ccf42bc79516 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:10.550695 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9f7deeff-07fe-4555-993a-ccf42bc79516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec74b5e25b114474859180dc3de642f9'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:10.557796 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9f7deeff-07fe-4555-993a-ccf42bc79516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec74b5e25b114474859180dc3de642f9'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:10.558477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9f7deeff-07fe-4555-993a-ccf42bc79516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec74b5e25b114474859180dc3de642f9'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:10.565275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9f7deeff-07fe-4555-993a-ccf42bc79516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec74b5e25b114474859180dc3de642f9'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:10.577049 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-9f7deeff-07fe-4555-993a-ccf42bc79516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec74b5e25b114474859180dc3de642f9'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:10.583280 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1417/2841] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:05:10 2022] DELETE /identity/v3/users/ec74b5e25b114474859180dc3de642f9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:10.861309 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:10.862789 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:10.876137 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d9273-82be-40f9-bf71-0015b7c4d0f0 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:10.935471 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8adba-7d02-46ea-8512-8d940811efd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:10.936244 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8adba-7d02-46ea-8512-8d940811efd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:10.936532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8adba-7d02-46ea-8512-8d940811efd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:10.936818 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8adba-7d02-46ea-8512-8d940811efd7 admin admin] PATH_INFO: `/v3/projects/c290f53bf156420182f9c2111573d805` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:10.938002 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8adba-7d02-46ea-8512-8d940811efd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c290f53bf156420182f9c2111573d805)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:10.940462 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8adba-7d02-46ea-8512-8d940811efd7 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:11.004951 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-11b8adba-7d02-46ea-8512-8d940811efd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c290f53bf156420182f9c2111573d805', 'request_id': 'req-11b8adba-7d02-46ea-8512-8d940811efd7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:11.010608 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-11b8adba-7d02-46ea-8512-8d940811efd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c290f53bf156420182f9c2111573d805', 'request_id': 'req-11b8adba-7d02-46ea-8512-8d940811efd7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:11.018504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-11b8adba-7d02-46ea-8512-8d940811efd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c290f53bf156420182f9c2111573d805', 'request_id': 'req-11b8adba-7d02-46ea-8512-8d940811efd7'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:11.029888 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-11b8adba-7d02-46ea-8512-8d940811efd7 admin admin] The token cache is being invalidate because project c290f53bf156420182f9c2111573d805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:05:11.030173 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-11b8adba-7d02-46ea-8512-8d940811efd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:11.030792 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-11b8adba-7d02-46ea-8512-8d940811efd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:11.032345 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1425/2842] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:05:10 2022] DELETE /identity/v3/projects/c290f53bf156420182f9c2111573d805 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:11.037921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546cfc8-fb8d-4801-91f0-aab7ef43b21e None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.038213 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546cfc8-fb8d-4801-91f0-aab7ef43b21e None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.038493 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546cfc8-fb8d-4801-91f0-aab7ef43b21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.056808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-6546cfc8-fb8d-4801-91f0-aab7ef43b21e None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:11.058008 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-6546cfc8-fb8d-4801-91f0-aab7ef43b21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:11.115700 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1418/2843] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:11.123508 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e011f-1909-4be5-9dfb-07b2781f87bf None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.123797 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e011f-1909-4be5-9dfb-07b2781f87bf None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.124077 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e011f-1909-4be5-9dfb-07b2781f87bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.142497 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-c81e011f-1909-4be5-9dfb-07b2781f87bf None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:11.144586 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c81e011f-1909-4be5-9dfb-07b2781f87bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:11.152562 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1426/2844] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:11.156754 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546cfc8-fb8d-4801-91f0-aab7ef43b21e None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:11.167135 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546cfc8-fb8d-4801-91f0-aab7ef43b21e None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:11.171694 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d67789-f22e-4e72-bcfc-24f74539c8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:11.172343 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67789-f22e-4e72-bcfc-24f74539c8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.172628 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67789-f22e-4e72-bcfc-24f74539c8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.172907 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67789-f22e-4e72-bcfc-24f74539c8c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.173631 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d67789-f22e-4e72-bcfc-24f74539c8c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86631) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 14:05:11.174089 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d67789-f22e-4e72-bcfc-24f74539c8c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:11.175726 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d67789-f22e-4e72-bcfc-24f74539c8c1 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:11.187173 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1419/2845] 10.222.0.52 () {58 vars in 1152 bytes} [Thu May 19 14:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 14:05:11.191743 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e011f-1909-4be5-9dfb-07b2781f87bf None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:11.197319 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:11.201216 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e011f-1909-4be5-9dfb-07b2781f87bf None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:11.208110 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5485e1-086e-4dba-a5de-6791e648b5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:11.208928 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5485e1-086e-4dba-a5de-6791e648b5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.209375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5485e1-086e-4dba-a5de-6791e648b5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.209823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5485e1-086e-4dba-a5de-6791e648b5c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.210524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5485e1-086e-4dba-a5de-6791e648b5c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:11.212460 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5485e1-086e-4dba-a5de-6791e648b5c5 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:11.238782 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1427/2846] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:05:11 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:05:11.242852 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d67789-f22e-4e72-bcfc-24f74539c8c1 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:11.252805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d67789-f22e-4e72-bcfc-24f74539c8c1 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:11.303297 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:11.312178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfe6dc-f8f6-46df-8ad8-862f52b5189f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:11.312866 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfe6dc-f8f6-46df-8ad8-862f52b5189f admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.313329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfe6dc-f8f6-46df-8ad8-862f52b5189f admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.313763 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfe6dc-f8f6-46df-8ad8-862f52b5189f admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.314758 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfe6dc-f8f6-46df-8ad8-862f52b5189f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:11.316722 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfe6dc-f8f6-46df-8ad8-862f52b5189f admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:11.336602 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1420/2847] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:05:11 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:05:11.340510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5485e1-086e-4dba-a5de-6791e648b5c5 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:11.351593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5485e1-086e-4dba-a5de-6791e648b5c5 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:11.356317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81910b49-deb7-482e-b1a0-e87b5909991b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:11.356974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81910b49-deb7-482e-b1a0-e87b5909991b admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.357277 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81910b49-deb7-482e-b1a0-e87b5909991b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.357538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81910b49-deb7-482e-b1a0-e87b5909991b admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.358268 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81910b49-deb7-482e-b1a0-e87b5909991b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:11.359887 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81910b49-deb7-482e-b1a0-e87b5909991b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:11.370891 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1428/2848] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:05:11.374831 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfe6dc-f8f6-46df-8ad8-862f52b5189f admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:11.385249 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfe6dc-f8f6-46df-8ad8-862f52b5189f admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:11.389889 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff85fae-4789-48cb-8295-4470584339b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:11.390549 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff85fae-4789-48cb-8295-4470584339b2 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.390879 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff85fae-4789-48cb-8295-4470584339b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.391167 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff85fae-4789-48cb-8295-4470584339b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.391895 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff85fae-4789-48cb-8295-4470584339b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:11.393480 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff85fae-4789-48cb-8295-4470584339b2 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:11.403648 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1421/2849] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 14:05:11.407903 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81910b49-deb7-482e-b1a0-e87b5909991b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:11.418338 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81910b49-deb7-482e-b1a0-e87b5909991b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:11.423033 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85d0e7-40d6-4872-aa95-90996573098c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:11.424510 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85d0e7-40d6-4872-aa95-90996573098c admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.424787 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85d0e7-40d6-4872-aa95-90996573098c admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.425069 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85d0e7-40d6-4872-aa95-90996573098c admin admin] PATH_INFO: `/v3/projects/e27ef7f3d5d34539bf4bcd8461bb8e60/users/0327a7804db84325bef2aa2a3c18aaae/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.439304 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85d0e7-40d6-4872-aa95-90996573098c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27ef7f3d5d34539bf4bcd8461bb8e60, user_id=0327a7804db84325bef2aa2a3c18aaae, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:11.442168 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85d0e7-40d6-4872-aa95-90996573098c admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:11.454828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1429/2850] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:05:11 2022] PUT /identity/v3/projects/e27ef7f3d5d34539bf4bcd8461bb8e60/users/0327a7804db84325bef2aa2a3c18aaae/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:05:11.466328 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530f042-bd3f-40a9-a13c-a31e95a16137 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.466827 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530f042-bd3f-40a9-a13c-a31e95a16137 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.466948 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530f042-bd3f-40a9-a13c-a31e95a16137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.493772 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-3530f042-bd3f-40a9-a13c-a31e95a16137 None None] MFA Rules not processed for user `0327a7804db84325bef2aa2a3c18aaae`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:11.495001 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-3530f042-bd3f-40a9-a13c-a31e95a16137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:11.560915 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1422/2851] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:11.565392 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85d0e7-40d6-4872-aa95-90996573098c admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:11.626218 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85d0e7-40d6-4872-aa95-90996573098c admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:11.631604 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b263c-7bce-4813-b0a8-9a5b81a3586b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:11.632251 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b263c-7bce-4813-b0a8-9a5b81a3586b admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.632538 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b263c-7bce-4813-b0a8-9a5b81a3586b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.632817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b263c-7bce-4813-b0a8-9a5b81a3586b admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.633772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b263c-7bce-4813-b0a8-9a5b81a3586b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:11.635695 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b263c-7bce-4813-b0a8-9a5b81a3586b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:11.660845 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1430/2852] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:05:11 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:05:11.664858 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530f042-bd3f-40a9-a13c-a31e95a16137 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:11.675792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530f042-bd3f-40a9-a13c-a31e95a16137 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:11.733318 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67379076-1c33-4e5a-a863-068698eee469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:11.733896 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67379076-1c33-4e5a-a863-068698eee469 admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.734179 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67379076-1c33-4e5a-a863-068698eee469 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.734458 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67379076-1c33-4e5a-a863-068698eee469 admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.735158 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67379076-1c33-4e5a-a863-068698eee469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:11.737776 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67379076-1c33-4e5a-a863-068698eee469 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:11.757661 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1423/2853] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:05:11 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 14:05:11.761859 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b263c-7bce-4813-b0a8-9a5b81a3586b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:11.772049 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b263c-7bce-4813-b0a8-9a5b81a3586b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:11.776648 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f17c03-cb75-41a0-a94a-289af95913c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:11.777297 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f17c03-cb75-41a0-a94a-289af95913c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.777590 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f17c03-cb75-41a0-a94a-289af95913c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.777857 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f17c03-cb75-41a0-a94a-289af95913c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.778943 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f17c03-cb75-41a0-a94a-289af95913c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:11.780186 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f17c03-cb75-41a0-a94a-289af95913c5 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:11.790734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1431/2854] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:05:11.795130 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67379076-1c33-4e5a-a863-068698eee469 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:11.805816 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67379076-1c33-4e5a-a863-068698eee469 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:11.810538 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098c6bc-10c5-4c17-9977-9d7ab8631f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:11.811236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098c6bc-10c5-4c17-9977-9d7ab8631f32 admin admin] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.811523 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098c6bc-10c5-4c17-9977-9d7ab8631f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.811817 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098c6bc-10c5-4c17-9977-9d7ab8631f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.812519 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098c6bc-10c5-4c17-9977-9d7ab8631f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:11.814106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098c6bc-10c5-4c17-9977-9d7ab8631f32 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:11.824634 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1424/2855] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:05:11.828697 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f17c03-cb75-41a0-a94a-289af95913c5 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:11.839058 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f17c03-cb75-41a0-a94a-289af95913c5 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:11.843808 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944c1f2-0905-463d-8924-ee99ddd08f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:11.844683 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944c1f2-0905-463d-8924-ee99ddd08f1e admin admin] REQUEST_METHOD: `PUT` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.844970 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944c1f2-0905-463d-8924-ee99ddd08f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.845250 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944c1f2-0905-463d-8924-ee99ddd08f1e admin admin] PATH_INFO: `/v3/projects/5dbc6c7abaeb451ab48b8676d23f132f/users/1aae155ba49843f4967792f508f2ec87/roles/a77e3ea4b46b4a3aaca8e814f53809b5` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.859617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944c1f2-0905-463d-8924-ee99ddd08f1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dbc6c7abaeb451ab48b8676d23f132f, user_id=1aae155ba49843f4967792f508f2ec87, role_id=a77e3ea4b46b4a3aaca8e814f53809b5)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:11.862741 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944c1f2-0905-463d-8924-ee99ddd08f1e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:11.875576 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1432/2856] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:05:11 2022] PUT /identity/v3/projects/5dbc6c7abaeb451ab48b8676d23f132f/users/1aae155ba49843f4967792f508f2ec87/roles/a77e3ea4b46b4a3aaca8e814f53809b5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:11.886433 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8415b-8ddc-4bf1-9592-046ed568e1d9 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:11.886727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8415b-8ddc-4bf1-9592-046ed568e1d9 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:11.887037 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8415b-8ddc-4bf1-9592-046ed568e1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:11.915324 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-e0c8415b-8ddc-4bf1-9592-046ed568e1d9 None None] MFA Rules not processed for user `1aae155ba49843f4967792f508f2ec87`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:11.916517 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e0c8415b-8ddc-4bf1-9592-046ed568e1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:11.948101 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:11.983178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1425/2857] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:11.988257 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944c1f2-0905-463d-8924-ee99ddd08f1e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:12.047378 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:12.052022 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944c1f2-0905-463d-8924-ee99ddd08f1e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:12.057899 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b123e-66d8-48e5-819c-6d52bf3ad0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:12.058555 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b123e-66d8-48e5-819c-6d52bf3ad0d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:12.058840 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b123e-66d8-48e5-819c-6d52bf3ad0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:12.059152 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b123e-66d8-48e5-819c-6d52bf3ad0d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:12.059993 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b123e-66d8-48e5-819c-6d52bf3ad0d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:12.061886 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b123e-66d8-48e5-819c-6d52bf3ad0d4 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:12.086360 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1433/2858] 10.222.0.52 () {60 vars in 1152 bytes} [Thu May 19 14:05:11 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:05:12.090371 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8415b-8ddc-4bf1-9592-046ed568e1d9 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:12.101403 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8415b-8ddc-4bf1-9592-046ed568e1d9 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:12.157038 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afc444-55c0-4708-a77d-bd29bd61894d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:12.157704 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afc444-55c0-4708-a77d-bd29bd61894d admin admin] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:12.157994 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afc444-55c0-4708-a77d-bd29bd61894d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:12.158336 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afc444-55c0-4708-a77d-bd29bd61894d admin admin] PATH_INFO: `/v3/users` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:12.159069 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afc444-55c0-4708-a77d-bd29bd61894d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:12.160970 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afc444-55c0-4708-a77d-bd29bd61894d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:12.180531 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1426/2859] 10.222.0.52 () {60 vars in 1143 bytes} [Thu May 19 14:05:12 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 14:05:12.184541 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b123e-66d8-48e5-819c-6d52bf3ad0d4 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:12.195652 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b123e-66d8-48e5-819c-6d52bf3ad0d4 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:12.200340 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dfd5f-267a-4e29-b7dc-16a3030b684e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:12.200999 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dfd5f-267a-4e29-b7dc-16a3030b684e admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:12.201289 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dfd5f-267a-4e29-b7dc-16a3030b684e admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:12.201569 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dfd5f-267a-4e29-b7dc-16a3030b684e admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:12.202376 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dfd5f-267a-4e29-b7dc-16a3030b684e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:12.203995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dfd5f-267a-4e29-b7dc-16a3030b684e admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:12.215122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1434/2860] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:05:12.219208 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afc444-55c0-4708-a77d-bd29bd61894d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:12.229350 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afc444-55c0-4708-a77d-bd29bd61894d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:12.234030 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b56840-fde4-4256-a7fb-ac5555dae34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:12.234938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b56840-fde4-4256-a7fb-ac5555dae34d admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:12.235363 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b56840-fde4-4256-a7fb-ac5555dae34d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:12.235643 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b56840-fde4-4256-a7fb-ac5555dae34d admin admin] PATH_INFO: `/v3/projects/1c9f9285f6b0410188f886d249b02a68/users/e7398b60f6574d8d8eeba8966f5838f3/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:12.250055 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b56840-fde4-4256-a7fb-ac5555dae34d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9f9285f6b0410188f886d249b02a68, user_id=e7398b60f6574d8d8eeba8966f5838f3, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:12.253186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b56840-fde4-4256-a7fb-ac5555dae34d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:12.267500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1427/2861] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:05:12 2022] PUT /identity/v3/projects/1c9f9285f6b0410188f886d249b02a68/users/e7398b60f6574d8d8eeba8966f5838f3/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:12.272034 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dfd5f-267a-4e29-b7dc-16a3030b684e admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:12.281796 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dfd5f-267a-4e29-b7dc-16a3030b684e admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:12.336466 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bebe1-5be4-4ed7-95ac-3b8be7752654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:12.337118 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bebe1-5be4-4ed7-95ac-3b8be7752654 admin admin] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:12.337425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bebe1-5be4-4ed7-95ac-3b8be7752654 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:12.337699 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bebe1-5be4-4ed7-95ac-3b8be7752654 admin admin] PATH_INFO: `/v3/roles` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:12.338416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bebe1-5be4-4ed7-95ac-3b8be7752654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:12.340012 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bebe1-5be4-4ed7-95ac-3b8be7752654 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:12.349431 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1435/2862] 10.222.0.52 () {58 vars in 1121 bytes} [Thu May 19 14:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 14:05:12.353701 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b56840-fde4-4256-a7fb-ac5555dae34d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:12.364782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b56840-fde4-4256-a7fb-ac5555dae34d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:12.369339 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b987461-eef9-47f6-8128-ff9b5be8a92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:12.370232 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b987461-eef9-47f6-8128-ff9b5be8a92d admin admin] REQUEST_METHOD: `PUT` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:12.370513 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b987461-eef9-47f6-8128-ff9b5be8a92d admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:12.370790 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b987461-eef9-47f6-8128-ff9b5be8a92d admin admin] PATH_INFO: `/v3/projects/1c9f9285f6b0410188f886d249b02a68/users/e7398b60f6574d8d8eeba8966f5838f3/roles/0893bb944dd94634accd3906aa391cd2` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:12.372782 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b987461-eef9-47f6-8128-ff9b5be8a92d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9f9285f6b0410188f886d249b02a68, user_id=e7398b60f6574d8d8eeba8966f5838f3, role_id=0893bb944dd94634accd3906aa391cd2)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:12.375428 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b987461-eef9-47f6-8128-ff9b5be8a92d admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:12.386906 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1428/2863] 10.222.0.52 () {60 vars in 1482 bytes} [Thu May 19 14:05:12 2022] PUT /identity/v3/projects/1c9f9285f6b0410188f886d249b02a68/users/e7398b60f6574d8d8eeba8966f5838f3/roles/0893bb944dd94634accd3906aa391cd2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:12.396532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a77b3d-ee51-4066-8891-c865d99c9b6d None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:12.396825 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a77b3d-ee51-4066-8891-c865d99c9b6d None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:12.397106 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a77b3d-ee51-4066-8891-c865d99c9b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:12.423668 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-c8a77b3d-ee51-4066-8891-c865d99c9b6d None None] MFA Rules not processed for user `e7398b60f6574d8d8eeba8966f5838f3`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:12.425402 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c8a77b3d-ee51-4066-8891-c865d99c9b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:12.492090 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1436/2864] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:05:12.504858 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b987461-eef9-47f6-8128-ff9b5be8a92d admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:12.506646 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5b987461-eef9-47f6-8128-ff9b5be8a92d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:12.564522 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b987461-eef9-47f6-8128-ff9b5be8a92d admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:12.568763 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa43d05-e9dc-4638-b02f-4089cf18bfb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:12.569395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa43d05-e9dc-4638-b02f-4089cf18bfb9 service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:12.569682 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa43d05-e9dc-4638-b02f-4089cf18bfb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:12.569952 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa43d05-e9dc-4638-b02f-4089cf18bfb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:12.607013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:12.626812 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa43d05-e9dc-4638-b02f-4089cf18bfb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:12.629140 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa43d05-e9dc-4638-b02f-4089cf18bfb9 service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:12.639225 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1429/2865] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:05:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:05:14.123048 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a77b3d-ee51-4066-8891-c865d99c9b6d None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:14.124473 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c8a77b3d-ee51-4066-8891-c865d99c9b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:14.137786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a77b3d-ee51-4066-8891-c865d99c9b6d None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:14.141369 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce9724-a1ba-4b74-8ca0-bff2c36744fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:14.142003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce9724-a1ba-4b74-8ca0-bff2c36744fa service cinder] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:14.142292 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce9724-a1ba-4b74-8ca0-bff2c36744fa service cinder] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:14.142660 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce9724-a1ba-4b74-8ca0-bff2c36744fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:14.192443 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:14.202231 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce9724-a1ba-4b74-8ca0-bff2c36744fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:14.203747 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce9724-a1ba-4b74-8ca0-bff2c36744fa service cinder] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:14.213227 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1437/2866] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:05:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:05:14.744593 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa43d05-e9dc-4638-b02f-4089cf18bfb9 service cinder] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:14.753606 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa43d05-e9dc-4638-b02f-4089cf18bfb9 service cinder] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:14.757282 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc44fe-e346-4d6c-94e8-6fea6e03a06e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ed0973226a2446ebf7c0cc3ee04e446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:14.758176 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc44fe-e346-4d6c-94e8-6fea6e03a06e service cinder] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:14.758462 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc44fe-e346-4d6c-94e8-6fea6e03a06e service cinder] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:14.758739 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc44fe-e346-4d6c-94e8-6fea6e03a06e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:14.768473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc44fe-e346-4d6c-94e8-6fea6e03a06e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:14.769880 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc44fe-e346-4d6c-94e8-6fea6e03a06e service cinder] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:14.779257 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1430/2867] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:05:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:05:18.617832 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce9724-a1ba-4b74-8ca0-bff2c36744fa service cinder] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:18.685141 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce9724-a1ba-4b74-8ca0-bff2c36744fa service cinder] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:18.689575 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:18.690435 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:18.690714 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:18.691018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] PATH_INFO: `/v3/users/0327a7804db84325bef2aa2a3c18aaae` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:18.692197 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=0327a7804db84325bef2aa2a3c18aaae)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:18.694126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:18.739826 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0327a7804db84325bef2aa2a3c18aaae'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:18.748126 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0327a7804db84325bef2aa2a3c18aaae'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:18.755674 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0327a7804db84325bef2aa2a3c18aaae'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:18.756324 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0327a7804db84325bef2aa2a3c18aaae'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:18.756934 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0327a7804db84325bef2aa2a3c18aaae'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:18.775985 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1438/2868] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:05:18 2022] DELETE /identity/v3/users/0327a7804db84325bef2aa2a3c18aaae => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:18.780450 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee525b-d999-4ede-a8ba-8b09d12dca13 None None] REQUEST_METHOD: `POST` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:18.780945 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee525b-d999-4ede-a8ba-8b09d12dca13 None None] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:18.781065 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee525b-d999-4ede-a8ba-8b09d12dca13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:18.798781 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.auth.core [None req-9fee525b-d999-4ede-a8ba-8b09d12dca13 None None] MFA Rules not processed for user `ea5484f975734ad68c3decb7b2d0c713`. Rule list: `[]` (Enabled: `True`). {{(pid=86631) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:18.799986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9fee525b-d999-4ede-a8ba-8b09d12dca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:18.859463 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1431/2869] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:18.869165 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:18.870602 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:18.928185 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfaa73f-e88a-4efc-bacf-39bd1ed321ec admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:18.931897 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26681904-fefb-4ad8-98b2-d3f907552c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:18.932535 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26681904-fefb-4ad8-98b2-d3f907552c9c service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:18.932817 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26681904-fefb-4ad8-98b2-d3f907552c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:18.933093 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26681904-fefb-4ad8-98b2-d3f907552c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:18.942182 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26681904-fefb-4ad8-98b2-d3f907552c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:18.943568 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26681904-fefb-4ad8-98b2-d3f907552c9c service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:18.953736 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1439/2870] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:05:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:05:19.220783 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee525b-d999-4ede-a8ba-8b09d12dca13 None None] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:19.222329 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-9fee525b-d999-4ede-a8ba-8b09d12dca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:19.237475 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee525b-d999-4ede-a8ba-8b09d12dca13 None None] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:19.242178 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:19.243097 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:19.243399 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:19.243894 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] PATH_INFO: `/v3/projects/e27ef7f3d5d34539bf4bcd8461bb8e60` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:19.245412 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e27ef7f3d5d34539bf4bcd8461bb8e60)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:19.247380 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:19.295850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e27ef7f3d5d34539bf4bcd8461bb8e60', 'request_id': 'req-f5570926-dd4a-4311-b17a-8ecc0da2458b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:19.301944 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e27ef7f3d5d34539bf4bcd8461bb8e60', 'request_id': 'req-f5570926-dd4a-4311-b17a-8ecc0da2458b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:19.310000 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e27ef7f3d5d34539bf4bcd8461bb8e60', 'request_id': 'req-f5570926-dd4a-4311-b17a-8ecc0da2458b'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:19.320730 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] The token cache is being invalidate because project e27ef7f3d5d34539bf4bcd8461bb8e60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:05:19.321013 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:19.322284 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:19.323871 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1432/2871] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:05:19 2022] DELETE /identity/v3/projects/e27ef7f3d5d34539bf4bcd8461bb8e60 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:19.328235 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26681904-fefb-4ad8-98b2-d3f907552c9c service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:19.338030 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26681904-fefb-4ad8-98b2-d3f907552c9c service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:19.346718 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:19.397177 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:19.398059 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:19.398347 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:19.398626 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] PATH_INFO: `/v3/users/1aae155ba49843f4967792f508f2ec87` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:19.399932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aae155ba49843f4967792f508f2ec87)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:19.402063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:19.447844 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aae155ba49843f4967792f508f2ec87'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:19.455107 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aae155ba49843f4967792f508f2ec87'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:19.462145 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aae155ba49843f4967792f508f2ec87'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:19.462950 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aae155ba49843f4967792f508f2ec87'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:19.463605 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aae155ba49843f4967792f508f2ec87'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:19.480856 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1440/2872] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:05:19 2022] DELETE /identity/v3/users/1aae155ba49843f4967792f508f2ec87 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:19.999117 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:20.000567 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:20.014938 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5570926-dd4a-4311-b17a-8ecc0da2458b admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:20.062551 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:20.076978 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:20.077903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:20.078133 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:20.078413 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] PATH_INFO: `/v3/projects/5dbc6c7abaeb451ab48b8676d23f132f` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:20.079631 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dbc6c7abaeb451ab48b8676d23f132f)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:20.081536 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:20.127793 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dbc6c7abaeb451ab48b8676d23f132f', 'request_id': 'req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.133993 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dbc6c7abaeb451ab48b8676d23f132f', 'request_id': 'req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.141537 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dbc6c7abaeb451ab48b8676d23f132f', 'request_id': 'req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.152564 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] The token cache is being invalidate because project 5dbc6c7abaeb451ab48b8676d23f132f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:05:20.152854 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.154177 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.155784 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1433/2873] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:05:19 2022] DELETE /identity/v3/projects/5dbc6c7abaeb451ab48b8676d23f132f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:20.159740 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:20.161134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:20.174506 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa508bce-df04-41ea-b969-a72d295ac54b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:20.230425 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:20.231317 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:20.231600 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:20.231874 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] PATH_INFO: `/v3/users/e7398b60f6574d8d8eeba8966f5838f3` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:20.233051 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7398b60f6574d8d8eeba8966f5838f3)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:20.235004 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:20.274146 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:20.284880 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7398b60f6574d8d8eeba8966f5838f3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.292102 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7398b60f6574d8d8eeba8966f5838f3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.299430 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7398b60f6574d8d8eeba8966f5838f3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.300074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7398b60f6574d8d8eeba8966f5838f3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.300682 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7398b60f6574d8d8eeba8966f5838f3'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.318080 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1441/2874] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:05:20 2022] DELETE /identity/v3/users/e7398b60f6574d8d8eeba8966f5838f3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:20.581051 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:20.582526 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:20.595477 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62177b3-6eef-4ad9-9ac5-077f88a4eb24 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:20.661048 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:20.661925 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:20.662217 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:20.662500 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] PATH_INFO: `/v3/projects/1c9f9285f6b0410188f886d249b02a68` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:20.663738 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c9f9285f6b0410188f886d249b02a68)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:20.666347 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:20.714096 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c9f9285f6b0410188f886d249b02a68', 'request_id': 'req-1874aa2e-b16b-4fea-8eca-b02ea3429c41'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.720212 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c9f9285f6b0410188f886d249b02a68', 'request_id': 'req-1874aa2e-b16b-4fea-8eca-b02ea3429c41'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.728270 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c9f9285f6b0410188f886d249b02a68', 'request_id': 'req-1874aa2e-b16b-4fea-8eca-b02ea3429c41'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.738697 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] The token cache is being invalidate because project 1c9f9285f6b0410188f886d249b02a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:05:20.739039 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.739663 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:05:20.741247 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1434/2875] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:05:20 2022] DELETE /identity/v3/projects/1c9f9285f6b0410188f886d249b02a68 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:05:25.814677 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:25.818074 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:25.886024 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5843a8-b1b7-4c83-8bcd-169861ea438f admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:25.944843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f5de6-325e-4019-84f9-df2a64d739b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:25.945505 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f5de6-325e-4019-84f9-df2a64d739b8 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:25.945786 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f5de6-325e-4019-84f9-df2a64d739b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:25.946063 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f5de6-325e-4019-84f9-df2a64d739b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:25.948678 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-8c1f5de6-325e-4019-84f9-df2a64d739b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:25.959811 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f5de6-325e-4019-84f9-df2a64d739b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:25.960837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f5de6-325e-4019-84f9-df2a64d739b8 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:25.970534 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1442/2876] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:05:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:05:30.735762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:35.966524 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=62,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:42.330344 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:42.332588 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:42.449154 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874aa2e-b16b-4fea-8eca-b02ea3429c41 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:42.458805 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddce785-29b9-4cf8-919f-1423ebc25671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:42.459515 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddce785-29b9-4cf8-919f-1423ebc25671 service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:42.459802 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddce785-29b9-4cf8-919f-1423ebc25671 service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:42.460081 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddce785-29b9-4cf8-919f-1423ebc25671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:42.462488 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2ddce785-29b9-4cf8-919f-1423ebc25671 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:42.475998 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddce785-29b9-4cf8-919f-1423ebc25671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:42.477667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddce785-29b9-4cf8-919f-1423ebc25671 service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:42.499159 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1435/2877] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:05:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:05:43.239525 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdfcc8-2575-4c9c-a04a-97408456904e None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:43.240067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdfcc8-2575-4c9c-a04a-97408456904e None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:43.240067 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdfcc8-2575-4c9c-a04a-97408456904e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:43.260593 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-0ffdfcc8-2575-4c9c-a04a-97408456904e None None] MFA Rules not processed for user `df54f67274d24463961e1f48dcb37447`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:05:43.262026 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-0ffdfcc8-2575-4c9c-a04a-97408456904e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:43.333667 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1443/2878] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:05:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 14:05:43.689903 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddce785-29b9-4cf8-919f-1423ebc25671 service nova] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:43.691437 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-2ddce785-29b9-4cf8-919f-1423ebc25671 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:43.703929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddce785-29b9-4cf8-919f-1423ebc25671 service nova] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:43.707540 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfae66c-300d-448d-ab32-2f556961ae1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:43.708192 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfae66c-300d-448d-ab32-2f556961ae1c service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:43.708473 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfae66c-300d-448d-ab32-2f556961ae1c service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:43.708753 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfae66c-300d-448d-ab32-2f556961ae1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:43.711040 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ccfae66c-300d-448d-ab32-2f556961ae1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:43.722712 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfae66c-300d-448d-ab32-2f556961ae1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:43.724188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfae66c-300d-448d-ab32-2f556961ae1c service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:43.732910 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1436/2879] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:05:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:05:45.228018 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdfcc8-2575-4c9c-a04a-97408456904e None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:05:45.229433 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-0ffdfcc8-2575-4c9c-a04a-97408456904e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:45.310775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdfcc8-2575-4c9c-a04a-97408456904e None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:05:45.315837 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5d6dc-54af-46c1-aaee-2b7461ba0eae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b105f8163ce84a9ba725e6b04d348446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:05:45.316491 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5d6dc-54af-46c1-aaee-2b7461ba0eae service placement] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:05:45.316799 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5d6dc-54af-46c1-aaee-2b7461ba0eae service placement] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:05:45.317079 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5d6dc-54af-46c1-aaee-2b7461ba0eae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:05:45.318875 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-e2a5d6dc-54af-46c1-aaee-2b7461ba0eae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:05:45.331532 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5d6dc-54af-46c1-aaee-2b7461ba0eae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:05:45.333742 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5d6dc-54af-46c1-aaee-2b7461ba0eae service placement] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:05:45.345654 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1444/2880] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:05:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:05:53.729024 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=40 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:05:55.340775 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=57 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:06:00.653114 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfae66c-300d-448d-ab32-2f556961ae1c service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:06:00.671045 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfae66c-300d-448d-ab32-2f556961ae1c service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:06:00.676886 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a43dd-75f4-477e-8a27-6e187ea4db6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:06:00.677539 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a43dd-75f4-477e-8a27-6e187ea4db6c service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:06:00.677825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a43dd-75f4-477e-8a27-6e187ea4db6c service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:06:00.678106 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a43dd-75f4-477e-8a27-6e187ea4db6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:06:00.680188 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-ba8a43dd-75f4-477e-8a27-6e187ea4db6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:06:00.693987 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a43dd-75f4-477e-8a27-6e187ea4db6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:06:00.695667 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a43dd-75f4-477e-8a27-6e187ea4db6c service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:06:00.707182 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1437/2881] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:06:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:06:10.702511 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=6 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:07:49.704852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d178c5-e761-445c-910e-144a7928f2fe None None] REQUEST_METHOD: `POST` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:07:49.705794 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d178c5-e761-445c-910e-144a7928f2fe None None] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:07:49.705794 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d178c5-e761-445c-910e-144a7928f2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:07:49.736707 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.auth.core [None req-03d178c5-e761-445c-910e-144a7928f2fe None None] MFA Rules not processed for user `df54f67274d24463961e1f48dcb37447`. Rule list: `[]` (Enabled: `True`). {{(pid=86630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 14:07:49.738129 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-03d178c5-e761-445c-910e-144a7928f2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:07:49.771416 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1445/2882] 10.222.0.52 () {58 vars in 957 bytes} [Thu May 19 14:07:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 14:07:50.234503 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a43dd-75f4-477e-8a27-6e187ea4db6c service nova] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:07:50.310599 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a43dd-75f4-477e-8a27-6e187ea4db6c service nova] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:07:50.317453 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1821affc-d11a-432c-bf0c-bad63668f0b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:07:50.318562 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821affc-d11a-432c-bf0c-bad63668f0b7 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:07:50.319095 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821affc-d11a-432c-bf0c-bad63668f0b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:07:50.319592 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821affc-d11a-432c-bf0c-bad63668f0b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:07:50.321825 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1821affc-d11a-432c-bf0c-bad63668f0b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:07:50.365149 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1821affc-d11a-432c-bf0c-bad63668f0b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:07:50.367443 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1821affc-d11a-432c-bf0c-bad63668f0b7 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:07:50.379934 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1438/2883] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:07:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:07:51.704810 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d178c5-e761-445c-910e-144a7928f2fe None None] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:07:51.707017 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-03d178c5-e761-445c-910e-144a7928f2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:07:51.779308 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d178c5-e761-445c-910e-144a7928f2fe None None] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:07:51.787350 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e30e79-a0ff-4dbf-9de2-83c76a465f5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b105f8163ce84a9ba725e6b04d348446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:07:51.788477 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30e79-a0ff-4dbf-9de2-83c76a465f5c service placement] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:07:51.788974 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30e79-a0ff-4dbf-9de2-83c76a465f5c service placement] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:07:51.789458 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e30e79-a0ff-4dbf-9de2-83c76a465f5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:07:51.793210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f7e30e79-a0ff-4dbf-9de2-83c76a465f5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:07:51.815044 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e30e79-a0ff-4dbf-9de2-83c76a465f5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:07:51.816744 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e30e79-a0ff-4dbf-9de2-83c76a465f5c service placement] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:07:51.832134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1446/2884] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:07:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:07:57.396607 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1821affc-d11a-432c-bf0c-bad63668f0b7 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:07:57.437762 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1821affc-d11a-432c-bf0c-bad63668f0b7 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:07:57.445930 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61e66-28ad-4009-9841-74d175bd618f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:07:57.446694 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61e66-28ad-4009-9841-74d175bd618f service nova] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:07:57.447015 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61e66-28ad-4009-9841-74d175bd618f service nova] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:07:57.447295 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61e66-28ad-4009-9841-74d175bd618f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:07:57.449303 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-b1a61e66-28ad-4009-9841-74d175bd618f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:07:57.460666 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a61e66-28ad-4009-9841-74d175bd618f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:07:57.462238 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a61e66-28ad-4009-9841-74d175bd618f service nova] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:07:57.473026 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1439/2885] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:07:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:08:01.825396 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=33 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:08:07.468275 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=36 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:38.556701 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e30e79-a0ff-4dbf-9de2-83c76a465f5c service placement] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:38.559236 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-f7e30e79-a0ff-4dbf-9de2-83c76a465f5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:38.597367 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e30e79-a0ff-4dbf-9de2-83c76a465f5c service placement] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:38.604764 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad4464-4c8a-41a4-a563-6ce70453d127 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6093d0c1fc41d98c58bace81e3d8e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:38.605807 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad4464-4c8a-41a4-a563-6ce70453d127 service nova] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:38.606167 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad4464-4c8a-41a4-a563-6ce70453d127 service nova] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:38.606478 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad4464-4c8a-41a4-a563-6ce70453d127 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:38.609843 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-1aad4464-4c8a-41a4-a563-6ce70453d127 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:38.624995 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad4464-4c8a-41a4-a563-6ce70453d127 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:38.626805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad4464-4c8a-41a4-a563-6ce70453d127 service nova] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:38.640371 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1447/2886] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:09:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 14:09:38.742395 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61e66-28ad-4009-9841-74d175bd618f service nova] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:38.754506 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61e66-28ad-4009-9841-74d175bd618f service nova] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:38.762447 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffabb56-1b50-4a6d-8c6e-1e52b7a5908d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:38.765492 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffabb56-1b50-4a6d-8c6e-1e52b7a5908d service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:38.765874 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffabb56-1b50-4a6d-8c6e-1e52b7a5908d service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:38.766166 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffabb56-1b50-4a6d-8c6e-1e52b7a5908d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:38.768727 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5ffabb56-1b50-4a6d-8c6e-1e52b7a5908d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:38.792894 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffabb56-1b50-4a6d-8c6e-1e52b7a5908d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:38.795075 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffabb56-1b50-4a6d-8c6e-1e52b7a5908d service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:38.814557 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1440/2887] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:09:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:09:40.026805 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad4464-4c8a-41a4-a563-6ce70453d127 service nova] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:40.036828 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad4464-4c8a-41a4-a563-6ce70453d127 service nova] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:40.040418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee5fa7-512d-4446-a805-4ea982fa8566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:40.042144 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee5fa7-512d-4446-a805-4ea982fa8566 service neutron] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:40.042144 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee5fa7-512d-4446-a805-4ea982fa8566 service neutron] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:40.042144 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee5fa7-512d-4446-a805-4ea982fa8566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:40.044023 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-afee5fa7-512d-4446-a805-4ea982fa8566 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:40.058751 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee5fa7-512d-4446-a805-4ea982fa8566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:40.060690 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee5fa7-512d-4446-a805-4ea982fa8566 service neutron] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:40.072597 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1448/2888] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:09:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:09:48.802107 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=6 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:50.067333 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=17 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:51.102792 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffabb56-1b50-4a6d-8c6e-1e52b7a5908d service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:51.104294 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-5ffabb56-1b50-4a6d-8c6e-1e52b7a5908d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:51.115029 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffabb56-1b50-4a6d-8c6e-1e52b7a5908d service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:51.118580 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c73fce-bcb7-45a1-a132-8158e5b06983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f18f635bff0142409e90da66547927f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:51.119256 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c73fce-bcb7-45a1-a132-8158e5b06983 service neutron] REQUEST_METHOD: `GET` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:51.119535 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c73fce-bcb7-45a1-a132-8158e5b06983 service neutron] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:51.119808 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c73fce-bcb7-45a1-a132-8158e5b06983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:51.122023 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-e2c73fce-bcb7-45a1-a132-8158e5b06983 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:51.209304 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c73fce-bcb7-45a1-a132-8158e5b06983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:51.210741 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c73fce-bcb7-45a1-a132-8158e5b06983 service neutron] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:51.230851 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1441/2889] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:09:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:09:58.213459 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee5fa7-512d-4446-a805-4ea982fa8566 service neutron] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:58.216005 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-afee5fa7-512d-4446-a805-4ea982fa8566 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:58.232603 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee5fa7-512d-4446-a805-4ea982fa8566 service neutron] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:58.237122 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:58.238003 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:58.238283 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:58.238617 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] PATH_INFO: `/v3/users/df54f67274d24463961e1f48dcb37447` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:58.239792 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df54f67274d24463961e1f48dcb37447)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:58.241689 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:58.293210 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df54f67274d24463961e1f48dcb37447'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.302905 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df54f67274d24463961e1f48dcb37447'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.310418 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df54f67274d24463961e1f48dcb37447'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.311134 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df54f67274d24463961e1f48dcb37447'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.311734 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df54f67274d24463961e1f48dcb37447'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.329527 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1449/2890] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:09:58 2022] DELETE /identity/v3/users/df54f67274d24463961e1f48dcb37447 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:09:58.548231 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c73fce-bcb7-45a1-a132-8158e5b06983 service neutron] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:58.558425 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c73fce-bcb7-45a1-a132-8158e5b06983 service neutron] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:58.622211 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:58.623186 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:58.623466 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:58.623737 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] PATH_INFO: `/v3/projects/6d87d3963b1b4fb29e09d8f72f64fb6e` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:58.624929 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d87d3963b1b4fb29e09d8f72f64fb6e)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:58.626817 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:58.675897 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d87d3963b1b4fb29e09d8f72f64fb6e', 'request_id': 'req-1b0a9027-318e-433d-a2d0-75f56da9fc24'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.681920 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d87d3963b1b4fb29e09d8f72f64fb6e', 'request_id': 'req-1b0a9027-318e-433d-a2d0-75f56da9fc24'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.691250 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d87d3963b1b4fb29e09d8f72f64fb6e', 'request_id': 'req-1b0a9027-318e-433d-a2d0-75f56da9fc24'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.696112 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:58.705723 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] The token cache is being invalidate because project 6d87d3963b1b4fb29e09d8f72f64fb6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:09:58.706011 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.706660 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.708308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1442/2891] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:09:58 2022] DELETE /identity/v3/projects/6d87d3963b1b4fb29e09d8f72f64fb6e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:09:58.712395 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:58.713852 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:58.727027 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f51bb-248f-44cd-ad66-b0b4bb8228e7 admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:58.787932 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:58.788814 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:58.789103 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:58.789382 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] PATH_INFO: `/v3/users/c8e8c5e2850d4274aa09fde41409b8d6` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:58.790673 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e8c5e2850d4274aa09fde41409b8d6)` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:58.792661 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:58.843754 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e8c5e2850d4274aa09fde41409b8d6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.852468 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e8c5e2850d4274aa09fde41409b8d6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.861614 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e8c5e2850d4274aa09fde41409b8d6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.862290 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e8c5e2850d4274aa09fde41409b8d6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.862934 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.notifications [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e8c5e2850d4274aa09fde41409b8d6'} {{(pid=86630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:58.873083 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:09:58.888823 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1450/2892] 10.222.0.52 () {58 vars in 1223 bytes} [Thu May 19 14:09:58 2022] DELETE /identity/v3/users/c8e8c5e2850d4274aa09fde41409b8d6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 14:09:59.076921 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] Authenticating user token {{(pid=86631) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:09:59.079113 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.fernet_utils [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86631) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:09:59.093244 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a9027-318e-433d-a2d0-75f56da9fc24 admin admin] Validating token access rules against request {{(pid=86631) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:09:59.156181 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386915b-a67c-4f78-8b85-dc76821b3c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea5484f975734ad68c3decb7b2d0c713', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f0d02746fbd4bebabb402313125d4b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86631) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:09:59.157063 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386915b-a67c-4f78-8b85-dc76821b3c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:09:59.157351 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386915b-a67c-4f78-8b85-dc76821b3c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:09:59.157625 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386915b-a67c-4f78-8b85-dc76821b3c63 admin admin] PATH_INFO: `/v3/projects/6eea54f5ac214cbeac589e8af91b3a84` {{(pid=86631) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:09:59.158850 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386915b-a67c-4f78-8b85-dc76821b3c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eea54f5ac214cbeac589e8af91b3a84)` {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:09:59.160824 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386915b-a67c-4f78-8b85-dc76821b3c63 admin admin] RBAC: Authorization granted {{(pid=86631) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:09:59.226986 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f386915b-a67c-4f78-8b85-dc76821b3c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eea54f5ac214cbeac589e8af91b3a84', 'request_id': 'req-f386915b-a67c-4f78-8b85-dc76821b3c63'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:59.238236 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f386915b-a67c-4f78-8b85-dc76821b3c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eea54f5ac214cbeac589e8af91b3a84', 'request_id': 'req-f386915b-a67c-4f78-8b85-dc76821b3c63'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:59.249641 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f386915b-a67c-4f78-8b85-dc76821b3c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eea54f5ac214cbeac589e8af91b3a84', 'request_id': 'req-f386915b-a67c-4f78-8b85-dc76821b3c63'} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:59.267006 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f386915b-a67c-4f78-8b85-dc76821b3c63 admin admin] The token cache is being invalidate because project 6eea54f5ac214cbeac589e8af91b3a84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86631) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 14:09:59.267308 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f386915b-a67c-4f78-8b85-dc76821b3c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:59.268069 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG keystone.notifications [None req-f386915b-a67c-4f78-8b85-dc76821b3c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86631) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 14:09:59.269671 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: [pid: 86631|app: 0|req: 1443/2893] 10.222.0.52 () {58 vars in 1232 bytes} [Thu May 19 14:09:59 2022] DELETE /identity/v3/projects/6eea54f5ac214cbeac589e8af91b3a84 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 14:10:08.887504 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:10:09.262571 dv-cinder-784815-14-iscsi devstack@keystone.service[86631]: DEBUG dbcounter [-] [86631] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86631) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 14:13:28.108375 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] Authenticating user token {{(pid=86630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 14:13:28.109829 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:13:28.184254 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006a6ee-0adf-4df1-aa07-fbec7533a21b admin admin] Validating token access rules against request {{(pid=86630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 14:13:28.246552 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b14726-0ca7-4879-9819-bf2ccd45564a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b105f8163ce84a9ba725e6b04d348446', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc6fd42a31942688f570f1ce86be510', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 14:13:28.247266 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b14726-0ca7-4879-9819-bf2ccd45564a service placement] REQUEST_METHOD: `GET` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 14:13:28.247548 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b14726-0ca7-4879-9819-bf2ccd45564a service placement] SCRIPT_NAME: `/identity` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 14:13:28.247833 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b14726-0ca7-4879-9819-bf2ccd45564a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 14:13:28.249816 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.fernet_utils [None req-c8b14726-0ca7-4879-9819-bf2ccd45564a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 14:13:28.262793 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b14726-0ca7-4879-9819-bf2ccd45564a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 14:13:28.264413 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b14726-0ca7-4879-9819-bf2ccd45564a service placement] RBAC: Authorization granted {{(pid=86630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 14:13:28.275579 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: [pid: 86630|app: 0|req: 1451/2894] 10.222.0.52 () {60 vars in 1365 bytes} [Thu May 19 14:13:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 14:13:38.270772 dv-cinder-784815-14-iscsi devstack@keystone.service[86630]: DEBUG dbcounter [-] [86630] Writing DB stats keystone:SELECT=58 {{(pid=86630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}